From 13fa805af28a3734e7bc4b1c63590d9d5b9de671 Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Tue, 2 Jun 2026 14:39:13 +0100 Subject: [PATCH 01/49] Bump version (#12773) --- aiohttp/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aiohttp/__init__.py b/aiohttp/__init__.py index b6afca45c60..5dfcd3841b9 100644 --- a/aiohttp/__init__.py +++ b/aiohttp/__init__.py @@ -1,4 +1,4 @@ -__version__ = "3.14.0" +__version__ = "3.14.1.dev0" from typing import TYPE_CHECKING From 9df2b300ad34c904b197af033ba6ba0349ca0517 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Tue, 2 Jun 2026 16:20:42 +0100 Subject: [PATCH 02/49] [PR #12774/c66e5cb4 backport][3.15] Point Dependabot to 3.15 branch (#12776) **This is a backport of PR #12774 as merged into master (c66e5cb4702b7e3a7373efe2848e35a61fe95e6e).** Co-authored-by: Sam Bull --- .github/dependabot.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 6ac64362454..fddd531da59 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -25,7 +25,7 @@ updates: directory: "/" labels: - dependencies - target-branch: "3.14" + target-branch: "3.15" schedule: interval: "daily" open-pull-requests-limit: 10 @@ -37,7 +37,7 @@ updates: - dependency-type: "all" labels: - dependencies - target-branch: "3.14" + target-branch: "3.15" schedule: interval: "daily" open-pull-requests-limit: 10 @@ -53,6 +53,6 @@ updates: directory: "/tests/autobahn/" labels: - dependencies - target-branch: "3.14" + target-branch: "3.15" schedule: interval: "monthly" From 3e25bb6707741a21c5aac60708abadfa848548e5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 2 Jun 2026 15:48:25 +0000 Subject: [PATCH 03/49] Bump click from 8.3.1 to 8.4.1 (#12779) Bumps [click](https://github.com/pallets/click) from 8.3.1 to 8.4.1.
Release notes

Sourced from click's releases.

8.4.1

This is the Click 8.4.1 fix release, which fixes bugs but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

PyPI: https://pypi.org/project/click/8.4.1/ Changes: https://click.palletsprojects.com/page/changes/#version-8-4-1 Milestone: https://github.com/pallets/click/milestone/32?closed=1

  • get_parameter_source() is available during eager callbacks and type conversion again. #3458 #3484
  • Zsh completion scripts parse correctly on Windows. #3277 # 3466
  • Shell completion of Choice Enum values produces a valid completion result. #3015
  • Fix empty byte-string handling in echo. #3487
  • Fix closed file error with echo_via_pager. #3449

8.4.0

This is the Click 8.4.0 feature release. A feature release may include new features, remove previously deprecated code, add new deprecation, or introduce potentially breaking changes.

We encourage everyone to upgrade. You can read more about our Version Support Policy on our website.

PyPI: https://pypi.org/project/click/8.4.0/ Changes: https://click.palletsprojects.com/page/changes/#version-8-4-0 Milestone https://github.com/pallets/click/milestone/30

  • ParamType typing improvements. #3371

    • :class:ParamType is now a generic abstract base class, parameterized by its converted value type.
    • :meth:~ParamType.convert return types are narrowed on all concrete types (str for :class:STRING, int for :class:INT, etc.).
    • :meth:~ParamType.to_info_dict returns specific :class:~typing.TypedDict subclasses instead of dict[str, Any].
    • :class:CompositeParamType and the number-range base are now generic with abstract methods.

  • Refactor convert_type to extract type inference into a private _guess_type helper, and add :func:typing.overload signatures. #3372

  • Parameter typing improvements. #2805

    • :class:Parameter is now an abstract base class, making explicit that it cannot be instantiated directly.
    • :attr:Parameter.name is now str instead of str | None. When expose_value=False, the name is set to "" instead of None.
    • The ctx parameter of :meth:Parameter.get_error_hint is now typed as Context | None, matching the runtime behavior.

  • Split string values from default_map for parameters with nargs > 1 or :class:Tuple type, matching environment variable behavior.

... (truncated)

Changelog

Sourced from click's changelog.

Version 8.4.1

Released 2026-05-21

  • get_parameter_source() is available during eager callbacks and type conversion again. :issue:3458 :issue:3484
  • Zsh completion scripts parse correctly on Windows. :issue:3277 :pr:3466
  • Shell completion of Choice Enum values produces a valid completion result. :issue:3015
  • Fix empty byte-string handling in echo. :issue:3487
  • Fix closed file error with echo_via_pager. :issue:3449

Version 8.4.0

Released 2026-05-17

  • :class:ParamType typing improvements. :pr:3371

    • :class:ParamType is now a generic abstract base class, parameterized by its converted value type.
    • :meth:~ParamType.convert return types are narrowed on all concrete types (str for :class:STRING, int for :class:INT, etc.).
    • :meth:~ParamType.to_info_dict returns specific :class:~typing.TypedDict subclasses instead of dict[str, Any].
    • :class:CompositeParamType and the number-range base are now generic with abstract methods.

  • Refactor convert_type to extract type inference into a private _guess_type helper, and add :func:typing.overload signatures. :pr:3372

  • :class:Parameter typing improvements. :pr:2805

    • :class:Parameter is now an abstract base class, making explicit that it cannot be instantiated directly.
    • :attr:Parameter.name is now str instead of str | None. When expose_value=False, the name is set to "" instead of None.
    • The ctx parameter of :meth:Parameter.get_error_hint is now typed as Context | None, matching the runtime behavior.

  • Split string values from default_map for parameters with nargs > 1 or :class:Tuple type, matching environment variable behavior. :issue:2745 :pr:3364

  • Auto-detect type=UNPROCESSED for flag_value of non-basic types (not str, int, float, or bool), so programmer-provided Python objects like classes and enum members are passed through unchanged instead of being stringified. Previously type=click.UNPROCESSED had to be set explicitly. :issue:2012 :pr:3363

... (truncated)

Commits
  • 6eeb50e release version 8.4.1
  • 67921d5 change log and doc fixes (#3495)
  • 9c41f46 Fix changelog and version admonitions
  • 6cb3477 fix skip condition
  • 5ee8e31 fix I/O operation on closed file error with CliRunner and echo_via_pager (#3482)
  • becbde5 pager doesn't close std streams
  • a5f5aa6 Handle empty bytes in echo (#3493)
  • 4d3db84 handle empty bytes in echo
  • d42f15b Fix get_parameter_source() during type conversion and eager callbacks (#3484)
  • 0baa8db Document ctx.params bypass with test and doc
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=click&package-manager=pip&previous-version=8.3.1&new-version=8.4.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/test-common-base.txt | 2 +- requirements/test-mobile.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements/test-common-base.txt b/requirements/test-common-base.txt index fc94dc894e0..1a8e58dcfce 100644 --- a/requirements/test-common-base.txt +++ b/requirements/test-common-base.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/test-common-base.txt --strip-extras requirements/test-common-base.in # -click==8.3.1 +click==8.4.1 # via wait-for-it coverage==7.14.1 # via pytest-cov diff --git a/requirements/test-mobile.txt b/requirements/test-mobile.txt index 9f18c58e8f6..84f3f397144 100644 --- a/requirements/test-mobile.txt +++ b/requirements/test-mobile.txt @@ -24,7 +24,7 @@ cffi==2.0.0 ; sys_platform != "android" and sys_platform != "ios" # via # -r requirements/test-mobile.in # pycares -click==8.4.0 +click==8.4.1 # via wait-for-it coverage==7.14.1 # via pytest-cov From d15bcd2e071d6663988aeeef8115b678039a346e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 2 Jun 2026 15:53:05 +0000 Subject: [PATCH 04/49] Bump virtualenv from 21.4.1 to 21.4.2 (#12777) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [virtualenv](https://github.com/pypa/virtualenv) from 21.4.1 to 21.4.2.
Release notes

Sourced from virtualenv's releases.

21.4.2

What's Changed

Full Changelog: https://github.com/pypa/virtualenv/compare/21.4.1...21.4.2

Changelog

Sourced from virtualenv's changelog.

Bugfixes - 21.4.2

  • Stop deactivate in the bash/zsh activation script from aborting under set -e when hash -r fails (for example with shell hashing disabled) by appending || true, matching CPython venv (gh-149701) and the existing non-deactivate call - by :user:gaborbernat. (:issue:3152)

v21.4.1 (2026-05-28)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=virtualenv&package-manager=pip&previous-version=21.4.1&new-version=21.4.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/lint.txt | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index f0d3f1efc64..6ac549e93fd 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -307,7 +307,7 @@ uvloop==0.22.1 ; platform_system != "Windows" # -r requirements/lint.in valkey==6.1.1 # via -r requirements/lint.in -virtualenv==21.4.1 +virtualenv==21.4.2 # via pre-commit wait-for-it==2.3.0 # via -r requirements/test-common-base.in diff --git a/requirements/dev.txt b/requirements/dev.txt index ab52c26ee9a..085217ed6a2 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -297,7 +297,7 @@ uvloop==0.22.1 ; platform_system != "Windows" and implementation_name == "cpytho # -r requirements/lint.in valkey==6.1.1 # via -r requirements/lint.in -virtualenv==21.4.1 +virtualenv==21.4.2 # via pre-commit wait-for-it==2.3.0 # via -r requirements/test-common-base.in diff --git a/requirements/lint.txt b/requirements/lint.txt index 036cb2d6a8b..5262a2483d8 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -128,7 +128,7 @@ uvloop==0.22.1 ; platform_system != "Windows" # via -r requirements/lint.in valkey==6.1.1 # via -r requirements/lint.in -virtualenv==21.4.1 +virtualenv==21.4.2 # via pre-commit zlib-ng==1.0.0 # via -r requirements/lint.in From 67f8adfe8dfc32e4f1f7bc4787ded9b20623ada5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 2 Jun 2026 15:57:25 +0000 Subject: [PATCH 05/49] Bump distlib from 0.4.0 to 0.4.1 (#12780) Bumps [distlib](https://github.com/pypa/distlib) from 0.4.0 to 0.4.1.
Changelog

Sourced from distlib's changelog.

0.4.1


Released: 2026-06-02

    

  • 

    scripts
    

      

    • Fix path traversal bug in handling entry points which allowed
escaping the scripts directory.
Thanks to tonghuaroot for the comprehensive report.
      • 

    

    • 

  • 

    tests
    

      

    • Fix #251:
Change test function following a reorganization which happened in the
Python stdlib.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=distlib&package-manager=pip&previous-version=0.4.0&new-version=0.4.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/lint.txt | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 6ac549e93fd..f036e7b44e9 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -64,7 +64,7 @@ cryptography==48.0.0 # via trustme cython==3.2.5 # via -r requirements/cython.in -distlib==0.4.0 +distlib==0.4.1 # via virtualenv docutils==0.21.2 # via diff --git a/requirements/dev.txt b/requirements/dev.txt index 085217ed6a2..f56fca6ea31 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -62,7 +62,7 @@ coverage==7.14.1 # pytest-cov cryptography==48.0.0 # via trustme -distlib==0.4.0 +distlib==0.4.1 # via virtualenv docutils==0.21.2 # via diff --git a/requirements/lint.txt b/requirements/lint.txt index 5262a2483d8..c4ca3798cb0 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -26,7 +26,7 @@ click==8.4.1 # via slotscheck cryptography==48.0.0 # via trustme -distlib==0.4.0 +distlib==0.4.1 # via virtualenv exceptiongroup==1.3.1 # via pytest From 80f85900f63552136ec6f12af52d47e8649f3efd Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 2 Jun 2026 16:08:27 +0000 Subject: [PATCH 06/49] Bump idna from 3.17 to 3.18 (#12782) Bumps [idna](https://github.com/kjd/idna) from 3.17 to 3.18.
Changelog

Sourced from idna's changelog.

3.18 (2026-06-02)

  • When decoding a domain, add a display argument that will pass through invalid labels rather than raising an exception.
Commits
  • f39ea90 Release 3.18
  • 40f4e40 Pre-release 3.18rc0
  • 1a5bf80 Merge pull request #253 from kjd/lenient-decode
  • 5bbb26f Merge branch 'master' into lenient-decode
  • c532bae Rename decode() lenient= option to display= (issue #248)
  • 0b1758b Merge pull request #252 from kjd/release-3.17
  • 47b5cde Add lenient option to decode() for best-effort label recovery (issue #248)
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=idna&package-manager=pip&previous-version=3.17&new-version=3.18)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/base-ft.txt | 2 +- requirements/base.txt | 2 +- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/doc-spelling.txt | 2 +- requirements/doc.txt | 2 +- requirements/lint.txt | 2 +- requirements/runtime-deps.txt | 2 +- requirements/test-common.txt | 2 +- requirements/test-ft.txt | 2 +- requirements/test-mobile.txt | 2 +- requirements/test.txt | 2 +- 12 files changed, 12 insertions(+), 12 deletions(-) diff --git a/requirements/base-ft.txt b/requirements/base-ft.txt index 26f04fd4011..be40c5bee8f 100644 --- a/requirements/base-ft.txt +++ b/requirements/base-ft.txt @@ -26,7 +26,7 @@ frozenlist==1.8.0 # aiosignal gunicorn==26.0.0 # via -r requirements/base-ft.in -idna==3.17 +idna==3.18 # via yarl multidict==6.7.1 # via diff --git a/requirements/base.txt b/requirements/base.txt index 8713fc6262c..5d8ba45de28 100644 --- a/requirements/base.txt +++ b/requirements/base.txt @@ -26,7 +26,7 @@ frozenlist==1.8.0 # aiosignal gunicorn==26.0.0 # via -r requirements/base.in -idna==3.17 +idna==3.18 # via yarl multidict==6.7.1 # via diff --git a/requirements/constraints.txt b/requirements/constraints.txt index f036e7b44e9..45f0baddaa1 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -92,7 +92,7 @@ gunicorn==26.0.0 # via -r requirements/base.in identify==2.6.19 # via pre-commit -idna==3.17 +idna==3.18 # via # requests # trustme diff --git a/requirements/dev.txt b/requirements/dev.txt index f56fca6ea31..b4bb6430326 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -90,7 +90,7 @@ gunicorn==26.0.0 # via -r requirements/base.in identify==2.6.19 # via pre-commit -idna==3.17 +idna==3.18 # via # requests # trustme diff --git a/requirements/doc-spelling.txt b/requirements/doc-spelling.txt index 2878edd94f6..e5c3306697f 100644 --- a/requirements/doc-spelling.txt +++ b/requirements/doc-spelling.txt @@ -20,7 +20,7 @@ docutils==0.21.2 # via # myst-parser # sphinx -idna==3.17 +idna==3.18 # via requests imagesize==2.0.0 # via sphinx diff --git a/requirements/doc.txt b/requirements/doc.txt index 2503191af64..d2a0f4e0b80 100644 --- a/requirements/doc.txt +++ b/requirements/doc.txt @@ -20,7 +20,7 @@ docutils==0.21.2 # via # myst-parser # sphinx -idna==3.17 +idna==3.18 # via requests imagesize==2.0.0 # via sphinx diff --git a/requirements/lint.txt b/requirements/lint.txt index c4ca3798cb0..3c64b3a496c 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -40,7 +40,7 @@ freezegun==1.5.5 # via -r requirements/lint.in identify==2.6.19 # via pre-commit -idna==3.17 +idna==3.18 # via trustme iniconfig==2.3.0 # via pytest diff --git a/requirements/runtime-deps.txt b/requirements/runtime-deps.txt index e86f12283b8..28afc656000 100644 --- a/requirements/runtime-deps.txt +++ b/requirements/runtime-deps.txt @@ -24,7 +24,7 @@ frozenlist==1.8.0 # via # -r requirements/runtime-deps.in # aiosignal -idna==3.17 +idna==3.18 # via yarl multidict==6.7.1 # via diff --git a/requirements/test-common.txt b/requirements/test-common.txt index 8d3bc4b479b..d3e2f267c3e 100644 --- a/requirements/test-common.txt +++ b/requirements/test-common.txt @@ -28,7 +28,7 @@ forbiddenfruit==0.1.4 # via blockbuster freezegun==1.5.5 # via -r requirements/test-common-base.in -idna==3.17 +idna==3.18 # via trustme iniconfig==2.3.0 # via pytest diff --git a/requirements/test-ft.txt b/requirements/test-ft.txt index f2fd489e532..d17865edc97 100644 --- a/requirements/test-ft.txt +++ b/requirements/test-ft.txt @@ -50,7 +50,7 @@ frozenlist==1.8.0 # aiosignal gunicorn==26.0.0 # via -r requirements/base-ft.in -idna==3.17 +idna==3.18 # via # trustme # yarl diff --git a/requirements/test-mobile.txt b/requirements/test-mobile.txt index 84f3f397144..907bc041b1d 100644 --- a/requirements/test-mobile.txt +++ b/requirements/test-mobile.txt @@ -38,7 +38,7 @@ frozenlist==1.8.0 # aiosignal gunicorn==26.0.0 # via -r requirements/base-ft.in -idna==3.17 +idna==3.18 # via yarl iniconfig==2.3.0 # via pytest diff --git a/requirements/test.txt b/requirements/test.txt index dcf60194f9f..aa15eeadcfb 100644 --- a/requirements/test.txt +++ b/requirements/test.txt @@ -50,7 +50,7 @@ frozenlist==1.8.0 # aiosignal gunicorn==26.0.0 # via -r requirements/base.in -idna==3.17 +idna==3.18 # via # trustme # yarl From 6acc4637a942e70029910b1f6014bdda0e391925 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 2 Jun 2026 19:29:24 +0000 Subject: [PATCH 07/49] Bump virtualenv from 21.4.1 to 21.4.2 (#12763) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [virtualenv](https://github.com/pypa/virtualenv) from 21.4.1 to 21.4.2.
Release notes

Sourced from virtualenv's releases.

21.4.2

What's Changed

Full Changelog: https://github.com/pypa/virtualenv/compare/21.4.1...21.4.2

Changelog

Sourced from virtualenv's changelog.

Bugfixes - 21.4.2

  • Stop deactivate in the bash/zsh activation script from aborting under set -e when hash -r fails (for example with shell hashing disabled) by appending || true, matching CPython venv (gh-149701) and the existing non-deactivate call - by :user:gaborbernat. (:issue:3152)

v21.4.1 (2026-05-28)

Commits

Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/lint.txt | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index f0d3f1efc64..6ac549e93fd 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -307,7 +307,7 @@ uvloop==0.22.1 ; platform_system != "Windows" # -r requirements/lint.in valkey==6.1.1 # via -r requirements/lint.in -virtualenv==21.4.1 +virtualenv==21.4.2 # via pre-commit wait-for-it==2.3.0 # via -r requirements/test-common-base.in diff --git a/requirements/dev.txt b/requirements/dev.txt index ab52c26ee9a..085217ed6a2 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -297,7 +297,7 @@ uvloop==0.22.1 ; platform_system != "Windows" and implementation_name == "cpytho # -r requirements/lint.in valkey==6.1.1 # via -r requirements/lint.in -virtualenv==21.4.1 +virtualenv==21.4.2 # via pre-commit wait-for-it==2.3.0 # via -r requirements/test-common-base.in diff --git a/requirements/lint.txt b/requirements/lint.txt index 036cb2d6a8b..5262a2483d8 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -128,7 +128,7 @@ uvloop==0.22.1 ; platform_system != "Windows" # via -r requirements/lint.in valkey==6.1.1 # via -r requirements/lint.in -virtualenv==21.4.1 +virtualenv==21.4.2 # via pre-commit zlib-ng==1.0.0 # via -r requirements/lint.in From 421b7917dc21974890f16aba89a871548ec9c0cb Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 2 Jun 2026 19:47:09 +0000 Subject: [PATCH 08/49] Bump click from 8.3.1 to 8.4.1 (#12771) Bumps [click](https://github.com/pallets/click) from 8.3.1 to 8.4.1.
Release notes

Sourced from click's releases.

8.4.1

This is the Click 8.4.1 fix release, which fixes bugs but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

PyPI: https://pypi.org/project/click/8.4.1/ Changes: https://click.palletsprojects.com/page/changes/#version-8-4-1 Milestone: https://github.com/pallets/click/milestone/32?closed=1

  • get_parameter_source() is available during eager callbacks and type conversion again. #3458 #3484
  • Zsh completion scripts parse correctly on Windows. #3277 # 3466
  • Shell completion of Choice Enum values produces a valid completion result. #3015
  • Fix empty byte-string handling in echo. #3487
  • Fix closed file error with echo_via_pager. #3449

8.4.0

This is the Click 8.4.0 feature release. A feature release may include new features, remove previously deprecated code, add new deprecation, or introduce potentially breaking changes.

We encourage everyone to upgrade. You can read more about our Version Support Policy on our website.

PyPI: https://pypi.org/project/click/8.4.0/ Changes: https://click.palletsprojects.com/page/changes/#version-8-4-0 Milestone https://github.com/pallets/click/milestone/30

  • ParamType typing improvements. #3371

    • :class:ParamType is now a generic abstract base class, parameterized by its converted value type.
    • :meth:~ParamType.convert return types are narrowed on all concrete types (str for :class:STRING, int for :class:INT, etc.).
    • :meth:~ParamType.to_info_dict returns specific :class:~typing.TypedDict subclasses instead of dict[str, Any].
    • :class:CompositeParamType and the number-range base are now generic with abstract methods.

  • Refactor convert_type to extract type inference into a private _guess_type helper, and add :func:typing.overload signatures. #3372

  • Parameter typing improvements. #2805

    • :class:Parameter is now an abstract base class, making explicit that it cannot be instantiated directly.
    • :attr:Parameter.name is now str instead of str | None. When expose_value=False, the name is set to "" instead of None.
    • The ctx parameter of :meth:Parameter.get_error_hint is now typed as Context | None, matching the runtime behavior.

  • Split string values from default_map for parameters with nargs > 1 or :class:Tuple type, matching environment variable behavior.

... (truncated)

Changelog

Sourced from click's changelog.

Version 8.4.1

Released 2026-05-21

  • get_parameter_source() is available during eager callbacks and type conversion again. :issue:3458 :issue:3484
  • Zsh completion scripts parse correctly on Windows. :issue:3277 :pr:3466
  • Shell completion of Choice Enum values produces a valid completion result. :issue:3015
  • Fix empty byte-string handling in echo. :issue:3487
  • Fix closed file error with echo_via_pager. :issue:3449

Version 8.4.0

Released 2026-05-17

  • :class:ParamType typing improvements. :pr:3371

    • :class:ParamType is now a generic abstract base class, parameterized by its converted value type.
    • :meth:~ParamType.convert return types are narrowed on all concrete types (str for :class:STRING, int for :class:INT, etc.).
    • :meth:~ParamType.to_info_dict returns specific :class:~typing.TypedDict subclasses instead of dict[str, Any].
    • :class:CompositeParamType and the number-range base are now generic with abstract methods.

  • Refactor convert_type to extract type inference into a private _guess_type helper, and add :func:typing.overload signatures. :pr:3372

  • :class:Parameter typing improvements. :pr:2805

    • :class:Parameter is now an abstract base class, making explicit that it cannot be instantiated directly.
    • :attr:Parameter.name is now str instead of str | None. When expose_value=False, the name is set to "" instead of None.
    • The ctx parameter of :meth:Parameter.get_error_hint is now typed as Context | None, matching the runtime behavior.

  • Split string values from default_map for parameters with nargs > 1 or :class:Tuple type, matching environment variable behavior. :issue:2745 :pr:3364

  • Auto-detect type=UNPROCESSED for flag_value of non-basic types (not str, int, float, or bool), so programmer-provided Python objects like classes and enum members are passed through unchanged instead of being stringified. Previously type=click.UNPROCESSED had to be set explicitly. :issue:2012 :pr:3363

... (truncated)

Commits
  • 6eeb50e release version 8.4.1
  • 67921d5 change log and doc fixes (#3495)
  • 9c41f46 Fix changelog and version admonitions
  • 6cb3477 fix skip condition
  • 5ee8e31 fix I/O operation on closed file error with CliRunner and echo_via_pager (#3482)
  • becbde5 pager doesn't close std streams
  • a5f5aa6 Handle empty bytes in echo (#3493)
  • 4d3db84 handle empty bytes in echo
  • d42f15b Fix get_parameter_source() during type conversion and eager callbacks (#3484)
  • 0baa8db Document ctx.params bypass with test and doc
  • Additional commits viewable in compare view

Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/test-common-base.txt | 2 +- requirements/test-mobile.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements/test-common-base.txt b/requirements/test-common-base.txt index fc94dc894e0..1a8e58dcfce 100644 --- a/requirements/test-common-base.txt +++ b/requirements/test-common-base.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/test-common-base.txt --strip-extras requirements/test-common-base.in # -click==8.3.1 +click==8.4.1 # via wait-for-it coverage==7.14.1 # via pytest-cov diff --git a/requirements/test-mobile.txt b/requirements/test-mobile.txt index 9f18c58e8f6..84f3f397144 100644 --- a/requirements/test-mobile.txt +++ b/requirements/test-mobile.txt @@ -24,7 +24,7 @@ cffi==2.0.0 ; sys_platform != "android" and sys_platform != "ios" # via # -r requirements/test-mobile.in # pycares -click==8.4.0 +click==8.4.1 # via wait-for-it coverage==7.14.1 # via pytest-cov From 5a5df7bc38ba53fa218f5209eb3c00b35d62d8fc Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Tue, 2 Jun 2026 23:55:17 +0100 Subject: [PATCH 09/49] [PR #12790/4ef04d66 backport][3.15] Readd codecov/project status check (#12791) **This is a backport of PR #12790 as merged into master (4ef04d66fa450156a5cc39c2ec4f00d4ca623d5b).** Co-authored-by: Sam Bull --- .codecov.yml | 3 --- 1 file changed, 3 deletions(-) diff --git a/.codecov.yml b/.codecov.yml index e3e81ac574d..fb11a3f13f9 100644 --- a/.codecov.yml +++ b/.codecov.yml @@ -10,9 +10,6 @@ comment: coverage: range: "95..100" - status: - project: no - component_management: individual_components: - component_id: project From c97baa30e9fde0ed25f07d6408c4bda793c5c9e6 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Wed, 3 Jun 2026 01:19:09 +0100 Subject: [PATCH 10/49] [PR #12787/4eb35886 backport][3.15] fix(connector): resolve race condition in TCPConnector.close() (#12793) **This is a backport of PR #12787 as merged into master (4eb358863b3797af1e5a6712b02a01f4a7fe985d).** --------- Co-authored-by: goingforstudying-ctrl Co-authored-by: Sam Bull --- CHANGES/12497.bugfix.rst | 1 + CONTRIBUTORS.txt | 1 + aiohttp/connector.py | 7 ++++-- tests/test_connector.py | 47 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 54 insertions(+), 2 deletions(-) create mode 100644 CHANGES/12497.bugfix.rst diff --git a/CHANGES/12497.bugfix.rst b/CHANGES/12497.bugfix.rst new file mode 100644 index 00000000000..7fd5883ccbd --- /dev/null +++ b/CHANGES/12497.bugfix.rst @@ -0,0 +1 @@ +Fixed a race condition in :py:class:`~aiohttp.TCPConnector` where closing the connector while a DNS resolution was in-flight could raise :py:exc:`AttributeError` instead of :py:exc:`~aiohttp.ClientConnectionError` -- by :user:`goingforstudying-ctrl`. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 24139ce2cda..830d86cb3a6 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -145,6 +145,7 @@ Gary Wilson Jr. Gene Hoffman Gennady Andreyev Georges Dubus +goingforstudying-ctrl Greg Holt Gregory Haynes Grigoriy Soldatov diff --git a/aiohttp/connector.py b/aiohttp/connector.py index 66cfaaca10a..b206f2e989a 100644 --- a/aiohttp/connector.py +++ b/aiohttp/connector.py @@ -1063,10 +1063,10 @@ async def close(self, *, abort_ssl: bool = False) -> None: - If ssl_shutdown_timeout=0: connections are aborted - If ssl_shutdown_timeout>0: graceful shutdown is performed """ - if self._resolver_owner: - await self._resolver.close() # Use abort_ssl param if explicitly set, otherwise use ssl_shutdown_timeout default await super().close(abort_ssl=abort_ssl or self._ssl_shutdown_timeout == 0) + if self._resolver_owner: + await self._resolver.close() @property def family(self) -> int: @@ -1109,6 +1109,9 @@ async def _resolve_host( for trace in traces: await trace.send_dns_resolvehost_start(host) + if self._closed: + raise ClientConnectionError("Connector is closed") + res = await self._resolver.resolve(host, port, family=self._family) if traces: diff --git a/tests/test_connector.py b/tests/test_connector.py index 15a2613f78b..27a7cc43ff6 100644 --- a/tests/test_connector.py +++ b/tests/test_connector.py @@ -24,6 +24,7 @@ import aiohttp from aiohttp import client, connector as connector_module, hdrs, web +from aiohttp.abc import AbstractResolver from aiohttp.client import ClientRequest, ClientTimeout from aiohttp.client_proto import ResponseHandler from aiohttp.client_reqrep import ConnectionKey @@ -4525,3 +4526,49 @@ async def test_connect_tunnel_connection_release( # Clean up to avoid resource warning conn.close() + + +async def test_tcp_connector_close_race_condition() -> None: + """Test closing TCPConnector while DNS resolution is in-flight.""" + loop = asyncio.get_running_loop() + resolve_started = loop.create_future() + close_started = loop.create_future() + + class FakeResolver(AbstractResolver): + async def resolve( + self, host: str, port: int = 0, family: int = socket.AF_INET + ) -> list[ResolveResult]: + resolve_started.set_result(None) + await close_started + return [ + { + "hostname": host, + "host": host, + "port": port, + "family": family, + "proto": 0, + "flags": socket.AI_NUMERICHOST, + } + ] + + async def close(self) -> None: + assert False + + connector = TCPConnector(use_dns_cache=False, resolver=FakeResolver()) + + async def resolve_host() -> None: + # The in-flight resolve should complete normally since close() + # happens after the resolver returns + result = await connector._resolve_host("localhost", 80) + assert len(result) == 1 + + async def close_connector() -> None: + await resolve_started + close_started.set_result(None) + await connector.close() + + await asyncio.gather(resolve_host(), close_connector()) + + # After close, new resolves should raise ClientConnectionError + with pytest.raises(aiohttp.ClientConnectionError, match="Connector is closed"): + await connector._resolve_host("localhost", 80) From 70fe11cc54ebb7a0cf08cb2be812d4d977839d9d Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Wed, 3 Jun 2026 01:19:28 +0100 Subject: [PATCH 11/49] [PR #12787/4eb35886 backport][3.14] fix(connector): resolve race condition in TCPConnector.close() (#12792) **This is a backport of PR #12787 as merged into master (4eb358863b3797af1e5a6712b02a01f4a7fe985d).** --------- Co-authored-by: goingforstudying-ctrl Co-authored-by: Sam Bull --- CHANGES/12497.bugfix.rst | 1 + CONTRIBUTORS.txt | 1 + aiohttp/connector.py | 7 ++++-- tests/test_connector.py | 47 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 54 insertions(+), 2 deletions(-) create mode 100644 CHANGES/12497.bugfix.rst diff --git a/CHANGES/12497.bugfix.rst b/CHANGES/12497.bugfix.rst new file mode 100644 index 00000000000..7fd5883ccbd --- /dev/null +++ b/CHANGES/12497.bugfix.rst @@ -0,0 +1 @@ +Fixed a race condition in :py:class:`~aiohttp.TCPConnector` where closing the connector while a DNS resolution was in-flight could raise :py:exc:`AttributeError` instead of :py:exc:`~aiohttp.ClientConnectionError` -- by :user:`goingforstudying-ctrl`. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 24139ce2cda..830d86cb3a6 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -145,6 +145,7 @@ Gary Wilson Jr. Gene Hoffman Gennady Andreyev Georges Dubus +goingforstudying-ctrl Greg Holt Gregory Haynes Grigoriy Soldatov diff --git a/aiohttp/connector.py b/aiohttp/connector.py index 66cfaaca10a..b206f2e989a 100644 --- a/aiohttp/connector.py +++ b/aiohttp/connector.py @@ -1063,10 +1063,10 @@ async def close(self, *, abort_ssl: bool = False) -> None: - If ssl_shutdown_timeout=0: connections are aborted - If ssl_shutdown_timeout>0: graceful shutdown is performed """ - if self._resolver_owner: - await self._resolver.close() # Use abort_ssl param if explicitly set, otherwise use ssl_shutdown_timeout default await super().close(abort_ssl=abort_ssl or self._ssl_shutdown_timeout == 0) + if self._resolver_owner: + await self._resolver.close() @property def family(self) -> int: @@ -1109,6 +1109,9 @@ async def _resolve_host( for trace in traces: await trace.send_dns_resolvehost_start(host) + if self._closed: + raise ClientConnectionError("Connector is closed") + res = await self._resolver.resolve(host, port, family=self._family) if traces: diff --git a/tests/test_connector.py b/tests/test_connector.py index 15a2613f78b..27a7cc43ff6 100644 --- a/tests/test_connector.py +++ b/tests/test_connector.py @@ -24,6 +24,7 @@ import aiohttp from aiohttp import client, connector as connector_module, hdrs, web +from aiohttp.abc import AbstractResolver from aiohttp.client import ClientRequest, ClientTimeout from aiohttp.client_proto import ResponseHandler from aiohttp.client_reqrep import ConnectionKey @@ -4525,3 +4526,49 @@ async def test_connect_tunnel_connection_release( # Clean up to avoid resource warning conn.close() + + +async def test_tcp_connector_close_race_condition() -> None: + """Test closing TCPConnector while DNS resolution is in-flight.""" + loop = asyncio.get_running_loop() + resolve_started = loop.create_future() + close_started = loop.create_future() + + class FakeResolver(AbstractResolver): + async def resolve( + self, host: str, port: int = 0, family: int = socket.AF_INET + ) -> list[ResolveResult]: + resolve_started.set_result(None) + await close_started + return [ + { + "hostname": host, + "host": host, + "port": port, + "family": family, + "proto": 0, + "flags": socket.AI_NUMERICHOST, + } + ] + + async def close(self) -> None: + assert False + + connector = TCPConnector(use_dns_cache=False, resolver=FakeResolver()) + + async def resolve_host() -> None: + # The in-flight resolve should complete normally since close() + # happens after the resolver returns + result = await connector._resolve_host("localhost", 80) + assert len(result) == 1 + + async def close_connector() -> None: + await resolve_started + close_started.set_result(None) + await connector.close() + + await asyncio.gather(resolve_host(), close_connector()) + + # After close, new resolves should raise ClientConnectionError + with pytest.raises(aiohttp.ClientConnectionError, match="Connector is closed"): + await connector._resolve_host("localhost", 80) From f5e823ba9fcb39cc3001f0129128d9d106c0a58a Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Wed, 3 Jun 2026 17:41:25 +0100 Subject: [PATCH 12/49] [PR #12796/a927da16 backport][3.15] Fix pipelined request after websocket rejection failing (#12800) **This is a backport of PR #12796 as merged into master (a927da1643b764bccfab91a0a683697b21333c05).** --------- Co-authored-by: Sam Bull --- aiohttp/web_protocol.py | 12 +++- tests/test_web_websocket_functional.py | 99 +++++++++++++++++++++++++- 2 files changed, 107 insertions(+), 4 deletions(-) diff --git a/aiohttp/web_protocol.py b/aiohttp/web_protocol.py index 69fe7b9bb29..d0719a7481d 100644 --- a/aiohttp/web_protocol.py +++ b/aiohttp/web_protocol.py @@ -422,7 +422,7 @@ def data_received(self, data: bytes) -> None: upgraded = False tail = b"" - for msg, payload in messages or (): + for msg, payload in messages: self._request_count += 1 self._messages.append((msg, payload)) @@ -703,8 +703,14 @@ async def finish_response( self._parser.set_upgraded(False) self._upgraded = False if self._message_tail: - self._parser.feed_data(self._message_tail) - self._message_tail = b"" + messages, _upgraded, tail = self._parser.feed_data(self._message_tail) + self._message_tail = tail + for msg, payload in messages: + self._request_count += 1 + self._messages.append((msg, payload)) + # This shouldn't be possible. If a future refactor results in this + # failing, then the code may need to be updated to set the waiter. + assert self._waiter is None try: prepare_meth = resp.prepare except AttributeError: diff --git a/tests/test_web_websocket_functional.py b/tests/test_web_websocket_functional.py index 7fd084af588..91862ec61fe 100644 --- a/tests/test_web_websocket_functional.py +++ b/tests/test_web_websocket_functional.py @@ -13,7 +13,7 @@ import aiohttp from aiohttp import hdrs, web from aiohttp.http import WSCloseCode, WSMsgType -from aiohttp.pytest_plugin import AiohttpClient +from aiohttp.pytest_plugin import AiohttpClient, AiohttpServer async def test_websocket_can_prepare(loop, aiohttp_client) -> None: @@ -32,6 +32,103 @@ async def handler(request): assert resp.status == 426 +async def test_pipelined_request_after_failed_websocket_upgrade( + aiohttp_server: AiohttpServer, +) -> None: + """Pipelined HTTP request runs after a declined websocket upgrade. + + The parser flips into upgraded mode when it sees the ``Upgrade`` + header and buffers any trailing bytes in ``_message_tail``. If the + handler declines the upgrade, ``finish_response()`` must replay the + tail through the parser so the pipelined request is dispatched + instead of stalling until the keep-alive timeout fires. + """ + + async def upgrade_handler(request: web.Request) -> NoReturn: + raise web.HTTPUpgradeRequired() + + async def second_handler(request: web.Request) -> web.Response: + return web.Response(text="second-ok") + + app = web.Application() + app.router.add_route("GET", "/", upgrade_handler) + app.router.add_route("GET", "/second", second_handler) + server = await aiohttp_server(app) + + # Need to use a raw writer in order to send the pipelined request. + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write( + b"GET / HTTP/1.1\r\n" + b"Host: localhost\r\n" + b"Upgrade: websocket\r\n" + b"Connection: Upgrade\r\n" + b"Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n" + b"Sec-WebSocket-Version: 13\r\n" + b"\r\n" + b"GET /second HTTP/1.1\r\n" + b"Host: localhost\r\n" + b"\r\n" + ) + await writer.drain() + + # Without the fix the second request is dropped and this read hangs + data = await asyncio.wait_for(reader.readuntil(b"second-ok"), timeout=5) + finally: + writer.close() + await writer.wait_closed() + + assert b"426" in data + + +async def test_partial_pipelined_request_after_failed_websocket_upgrade( + aiohttp_server: AiohttpServer, +) -> None: + """Partial pipelined bytes are preserved across finish_response. + + Only part of the second request rides along with the upgrade, so + feed_data() in finish_response() returns no messages and the parser + keeps the partial bytes in its internal buffer. When the remainder + arrives via data_received() the request is completed and dispatched. + """ + + async def upgrade_handler(request: web.Request) -> NoReturn: + raise web.HTTPUpgradeRequired() + + async def second_handler(request: web.Request) -> web.Response: + return web.Response(text="second-ok") + + app = web.Application() + app.router.add_route("GET", "/", upgrade_handler) + app.router.add_route("GET", "/second", second_handler) + server = await aiohttp_server(app) + + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write( + b"GET / HTTP/1.1\r\n" + b"Host: localhost\r\n" + b"Upgrade: websocket\r\n" + b"Connection: Upgrade\r\n" + b"Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n" + b"Sec-WebSocket-Version: 13\r\n" + b"\r\n" + b"GET /second HTT" # truncated mid-request + ) + await writer.drain() + + first = await asyncio.wait_for(reader.readuntil(b"\r\n\r\n"), timeout=5) + assert b"426" in first + + writer.write(b"P/1.1\r\nHost: localhost\r\n\r\n") + await writer.drain() + + await asyncio.wait_for(reader.readuntil(b"second-ok"), timeout=5) + finally: + writer.close() + await writer.wait_closed() + + async def test_handshake_connection_header_substring_not_a_token( aiohttp_client: AiohttpClient, ) -> None: From 8943d34337c133aa2a33f1fd10a30950d8dcb20e Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Wed, 3 Jun 2026 17:55:01 +0100 Subject: [PATCH 13/49] [PR #12796/a927da16 backport][3.14] Fix pipelined request after websocket rejection failing (#12799) **This is a backport of PR #12796 as merged into master (a927da1643b764bccfab91a0a683697b21333c05).** --------- Co-authored-by: Sam Bull --- aiohttp/web_protocol.py | 12 +++- tests/test_web_websocket_functional.py | 99 +++++++++++++++++++++++++- 2 files changed, 107 insertions(+), 4 deletions(-) diff --git a/aiohttp/web_protocol.py b/aiohttp/web_protocol.py index 69fe7b9bb29..d0719a7481d 100644 --- a/aiohttp/web_protocol.py +++ b/aiohttp/web_protocol.py @@ -422,7 +422,7 @@ def data_received(self, data: bytes) -> None: upgraded = False tail = b"" - for msg, payload in messages or (): + for msg, payload in messages: self._request_count += 1 self._messages.append((msg, payload)) @@ -703,8 +703,14 @@ async def finish_response( self._parser.set_upgraded(False) self._upgraded = False if self._message_tail: - self._parser.feed_data(self._message_tail) - self._message_tail = b"" + messages, _upgraded, tail = self._parser.feed_data(self._message_tail) + self._message_tail = tail + for msg, payload in messages: + self._request_count += 1 + self._messages.append((msg, payload)) + # This shouldn't be possible. If a future refactor results in this + # failing, then the code may need to be updated to set the waiter. + assert self._waiter is None try: prepare_meth = resp.prepare except AttributeError: diff --git a/tests/test_web_websocket_functional.py b/tests/test_web_websocket_functional.py index 7fd084af588..91862ec61fe 100644 --- a/tests/test_web_websocket_functional.py +++ b/tests/test_web_websocket_functional.py @@ -13,7 +13,7 @@ import aiohttp from aiohttp import hdrs, web from aiohttp.http import WSCloseCode, WSMsgType -from aiohttp.pytest_plugin import AiohttpClient +from aiohttp.pytest_plugin import AiohttpClient, AiohttpServer async def test_websocket_can_prepare(loop, aiohttp_client) -> None: @@ -32,6 +32,103 @@ async def handler(request): assert resp.status == 426 +async def test_pipelined_request_after_failed_websocket_upgrade( + aiohttp_server: AiohttpServer, +) -> None: + """Pipelined HTTP request runs after a declined websocket upgrade. + + The parser flips into upgraded mode when it sees the ``Upgrade`` + header and buffers any trailing bytes in ``_message_tail``. If the + handler declines the upgrade, ``finish_response()`` must replay the + tail through the parser so the pipelined request is dispatched + instead of stalling until the keep-alive timeout fires. + """ + + async def upgrade_handler(request: web.Request) -> NoReturn: + raise web.HTTPUpgradeRequired() + + async def second_handler(request: web.Request) -> web.Response: + return web.Response(text="second-ok") + + app = web.Application() + app.router.add_route("GET", "/", upgrade_handler) + app.router.add_route("GET", "/second", second_handler) + server = await aiohttp_server(app) + + # Need to use a raw writer in order to send the pipelined request. + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write( + b"GET / HTTP/1.1\r\n" + b"Host: localhost\r\n" + b"Upgrade: websocket\r\n" + b"Connection: Upgrade\r\n" + b"Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n" + b"Sec-WebSocket-Version: 13\r\n" + b"\r\n" + b"GET /second HTTP/1.1\r\n" + b"Host: localhost\r\n" + b"\r\n" + ) + await writer.drain() + + # Without the fix the second request is dropped and this read hangs + data = await asyncio.wait_for(reader.readuntil(b"second-ok"), timeout=5) + finally: + writer.close() + await writer.wait_closed() + + assert b"426" in data + + +async def test_partial_pipelined_request_after_failed_websocket_upgrade( + aiohttp_server: AiohttpServer, +) -> None: + """Partial pipelined bytes are preserved across finish_response. + + Only part of the second request rides along with the upgrade, so + feed_data() in finish_response() returns no messages and the parser + keeps the partial bytes in its internal buffer. When the remainder + arrives via data_received() the request is completed and dispatched. + """ + + async def upgrade_handler(request: web.Request) -> NoReturn: + raise web.HTTPUpgradeRequired() + + async def second_handler(request: web.Request) -> web.Response: + return web.Response(text="second-ok") + + app = web.Application() + app.router.add_route("GET", "/", upgrade_handler) + app.router.add_route("GET", "/second", second_handler) + server = await aiohttp_server(app) + + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write( + b"GET / HTTP/1.1\r\n" + b"Host: localhost\r\n" + b"Upgrade: websocket\r\n" + b"Connection: Upgrade\r\n" + b"Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n" + b"Sec-WebSocket-Version: 13\r\n" + b"\r\n" + b"GET /second HTT" # truncated mid-request + ) + await writer.drain() + + first = await asyncio.wait_for(reader.readuntil(b"\r\n\r\n"), timeout=5) + assert b"426" in first + + writer.write(b"P/1.1\r\nHost: localhost\r\n\r\n") + await writer.drain() + + await asyncio.wait_for(reader.readuntil(b"second-ok"), timeout=5) + finally: + writer.close() + await writer.wait_closed() + + async def test_handshake_connection_header_substring_not_a_token( aiohttp_client: AiohttpClient, ) -> None: From 4cc390a68004cb4b1640c91d4e3c802fcc124a94 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 4 Jun 2026 01:56:10 +0000 Subject: [PATCH 14/49] Bump astral-sh/setup-uv from 8.1.0 to 8.2.0 (#12802) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.1.0 to 8.2.0.
Release notes

Sourced from astral-sh/setup-uv's releases.

v8.2.0 🌈 New inputs quiet and download-from-astral-mirror

Changes

This release brings two new inputs and a few bug fixes.

New inputs

Lets talk about the new inputs first.

quiet

Pretty simple. It turns of all info loggings. Useful if you use this in a composite action and are not interested in all the details. In the upcoming releases we will add log groups to fully implement support for "less noise"

[!NOTE]
Warnings and errors are always logged.

download-from-astral-mirror

In some cases you may want to directly use the fallback of checking for available versions and downloading releases from GitHub instead of using the astral.sh mirror. Setting download-from-astral-mirror: false allows you to do that.

Bugfixes

When using the astral.sh mirror to query available versions and download releases (done by default) we now stop sending the GitHub token in the header. The mirror never looked at it but we shouldn't be handing out that data even if it is just a short lived token. All other bugfixes try to limit the impact of failed GitHub queries due to retries and other faults.

We couldn't pinpoint all rootcauses yet but added more logging for error cases to track them down.

🐛 Bug fixes

🚀 Enhancements

🧰 Maintenance

... (truncated)

Commits
  • fac544c chore(deps): roll up dependabot updates (#903)
  • 7390f77 docs: update dependabot rollup biome guidance (#902)
  • 363c64a chore(deps): roll up dependabot updates (#901)
  • c4fcbaf chore(deps): bump release-drafter/release-drafter from 7.3.0 to 7.3.1 (#900)
  • 8e642c5 chore: update known checksums for 0.11.18 (#899)
  • a92cb43 Add quiet input to suppress info-level log output (#898)
  • e07f2ac chore(deps): bump eifinger/actionlint-action from 1.10.1 to 1.10.2 (#842)
  • bc4034e chore(deps): bump github/codeql-action from 4.35.4 to 4.36.0 (#893)
  • df42d4f chore(deps): bump zizmorcore/zizmor-action from 0.5.5 to 0.5.6 (#891)
  • b9c8c4c feat: add download-from-astral-mirror input (#897)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=astral-sh/setup-uv&package-manager=github_actions&previous-version=8.1.0&new-version=8.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/ci-cd.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci-cd.yml b/.github/workflows/ci-cd.yml index 9bb01cb551c..a6ae26a7134 100644 --- a/.github/workflows/ci-cd.yml +++ b/.github/workflows/ci-cd.yml @@ -175,7 +175,7 @@ jobs: # important: do not use system python env: UV_PYTHON_PREFERENCE: only-managed - uses: astral-sh/setup-uv@v8.1.0 + uses: astral-sh/setup-uv@v8.2.0 with: python-version: ${{ matrix.pyver }} activate-environment: true @@ -282,7 +282,7 @@ jobs: # important: do not use system python env: UV_PYTHON_PREFERENCE: only-managed - uses: astral-sh/setup-uv@v8.1.0 + uses: astral-sh/setup-uv@v8.2.0 with: python-version: ${{ matrix.pyver }} activate-environment: true @@ -332,7 +332,7 @@ jobs: # important: do not use system python env: UV_PYTHON_PREFERENCE: only-managed - uses: astral-sh/setup-uv@v8.1.0 + uses: astral-sh/setup-uv@v8.2.0 with: python-version: ${{ matrix.pyver }} activate-environment: true From 5079abc87d742fec2f1513c0eb7a7e00f737c928 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 4 Jun 2026 02:14:43 +0000 Subject: [PATCH 15/49] Bump snowballstemmer from 3.1.0 to 3.1.1 (#12805) Bumps [snowballstemmer](https://github.com/snowballstem/snowball) from 3.1.0 to 3.1.1.
Changelog

Sourced from snowballstemmer's changelog.

Snowball 3.1.1 (2026-06-03)

Compiler changes

  • Bug fixes:

    • Fix a segmentation fault after reporting an error for a string command not followed by a string variable name or string literal. Bug introduced in 3.1.0. Patch from Jerry James (#287).

  • Compiler command-line options:

    • Emit an error for -o -/-output -. Output to stdout is not supported because we need to generate multiple files for some target languages. We were interpreting - as a base filename to append extensions to, so we'd create -.c and -.h for C, but creating filenames that start with - seems unhelpful.

Generic code generation changes

  • Bug fixes:

    • Variable localisation was failing to check the expression on the RHS of an integer test for uses of a variable, so could incorrectly localise an integer variable whose value should have persisted between calls to a function. This bug won't realistically manifest in real world Snowball code.

  • Optimisations:

    • Inline some routines which are only used once. This is done for routines consisting of a single non-compound command (or cases such as not <boolean> and goto <grouping> which we internally synthesise a non-compound command for). Localisation of variables happens after inlining, so variables can now be localised in more cases.

    • test next and not next are both now simplified to a comparison between cursor and limit (like not atlimit and atlimit). We already normalise hop 1 to next, so test hop 1 and not hop 1 are also simplified in this way.

    • Simplify not applied to an integer test by removing the not and flipping the sense of the test (e.g. not $(x > y) becomes $(x <= y)) which results in simpler generated code. More usefully in real world code, this also results in simpler generated code for not atlimit (since atlimit is converted $(cursor >= limit) or $(cursor <= limit) (depending on the current direction).

... (truncated)

Commits
  • cd195b5 Update for 3.1.1
  • 80d885c NEWS: Update draft entry
  • 5346c74 C++: Compile runtime as C++
  • 537f970 python: Add classifier for 3.14
  • 3aeb013 python: Skip Natural Language :: Sesotho classifier
  • 91fa20b NEWS: Update draft entry
  • a04abeb Don't use extern "C" for functions which can throw
  • 49bb623 C++: Hook up properly
  • 7ab50ed CI: Fix coverage job
  • 5f0a33c NEWS: Add draft entry
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=snowballstemmer&package-manager=pip&previous-version=3.1.0&new-version=3.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/doc-spelling.txt | 2 +- requirements/doc.txt | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 45f0baddaa1..4118e9578a9 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -239,7 +239,7 @@ six==1.17.0 # via python-dateutil slotscheck==0.20.0 # via -r requirements/lint.in -snowballstemmer==3.1.0 +snowballstemmer==3.1.1 # via sphinx sphinx==8.1.3 # via diff --git a/requirements/dev.txt b/requirements/dev.txt index b4bb6430326..436eebebc7d 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -232,7 +232,7 @@ six==1.17.0 # via python-dateutil slotscheck==0.20.0 # via -r requirements/lint.in -snowballstemmer==3.1.0 +snowballstemmer==3.1.1 # via sphinx sphinx==8.1.3 # via diff --git a/requirements/doc-spelling.txt b/requirements/doc-spelling.txt index e5c3306697f..3380097a3d9 100644 --- a/requirements/doc-spelling.txt +++ b/requirements/doc-spelling.txt @@ -56,7 +56,7 @@ requests==2.34.2 # via # sphinx # sphinxcontrib-spelling -snowballstemmer==3.1.0 +snowballstemmer==3.1.1 # via sphinx sphinx==8.1.3 # via diff --git a/requirements/doc.txt b/requirements/doc.txt index d2a0f4e0b80..32758208e3d 100644 --- a/requirements/doc.txt +++ b/requirements/doc.txt @@ -52,7 +52,7 @@ pyyaml==6.0.3 # sphinxcontrib-mermaid requests==2.34.2 # via sphinx -snowballstemmer==3.1.0 +snowballstemmer==3.1.1 # via sphinx sphinx==8.1.3 # via From b927ecec944877ebd21c54c66919c21a91fadd89 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Thu, 4 Jun 2026 18:59:04 +0000 Subject: [PATCH 16/49] [PR #12809/5771a85f backport][3.15] Revert "Move pip-tools into pyproject.toml" (#12811) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- .pip-tools.toml | 5 +++++ pyproject.toml | 7 ------- 2 files changed, 5 insertions(+), 7 deletions(-) create mode 100644 .pip-tools.toml diff --git a/.pip-tools.toml b/.pip-tools.toml new file mode 100644 index 00000000000..86969d6eda2 --- /dev/null +++ b/.pip-tools.toml @@ -0,0 +1,5 @@ +[pip-tools] +allow-unsafe = false +resolver = "backtracking" +strip-extras = true +unsafe-package = "aiohttp" diff --git a/pyproject.toml b/pyproject.toml index 4011ba49ce2..bf490bf7cdc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -175,13 +175,6 @@ skip = "pp*" # iOS currently does not support build[uv] build-frontend = "build" -[tool.pip-tools] -# Dependabot won't pick up .pip-tools.toml, so must be defined here. -allow-unsafe = false -resolver = "backtracking" -strip-extras = true -unsafe-package = ["aiohttp"] - [tool.codespell] skip = '.git,*.pdf,*.svg,Makefile,CONTRIBUTORS.txt,venvs,_build' ignore-words-list = 'te,ue' From b1458fe10e0e879c37ba244c42945e160b6a906b Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Thu, 4 Jun 2026 22:47:16 +0100 Subject: [PATCH 17/49] Close transport on BaseException in ResponseHandler.data_received (#12798) (#12812) (cherry picked from commit 24cb57bb8755ac795224db2e24d5baf6c4735812) --- CHANGES/12795.bugfix.rst | 1 + aiohttp/client_proto.py | 4 +++- tests/test_client_proto.py | 29 +++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 1 deletion(-) create mode 100644 CHANGES/12795.bugfix.rst diff --git a/CHANGES/12795.bugfix.rst b/CHANGES/12795.bugfix.rst new file mode 100644 index 00000000000..d08ea779287 --- /dev/null +++ b/CHANGES/12795.bugfix.rst @@ -0,0 +1 @@ +Fixed ``CancelledError`` not closing a connection -- by :user:`aiolibsbot`. diff --git a/aiohttp/client_proto.py b/aiohttp/client_proto.py index e996d43eb2e..a0b8512af9b 100644 --- a/aiohttp/client_proto.py +++ b/aiohttp/client_proto.py @@ -322,12 +322,14 @@ def data_received(self, data: bytes) -> None: # parse http messages try: messages, upgraded, tail = self._parser.feed_data(data) - except Exception as underlying_exc: + except BaseException as underlying_exc: if self.transport is not None: # connection.release() could be called BEFORE # data_received(), the transport is already # closed in this case self.transport.close() + if not isinstance(underlying_exc, Exception): + raise # should_close is True after the call if isinstance(underlying_exc, HttpProcessingError): exc = HttpProcessingError( diff --git a/tests/test_client_proto.py b/tests/test_client_proto.py index ed6e602b0d5..b4cd15b00f8 100644 --- a/tests/test_client_proto.py +++ b/tests/test_client_proto.py @@ -1,6 +1,7 @@ import asyncio from unittest import mock +import pytest from yarl import URL from aiohttp import http @@ -164,6 +165,34 @@ class PatchableHttpResponseParser(http.HttpResponseParser): assert isinstance(proto.exception(), http.HttpProcessingError) +async def test_base_exception_during_data_received_closes_transport() -> None: + loop = asyncio.get_running_loop() + proto = ResponseHandler(loop=loop) + + class PatchableHttpResponseParser(http.HttpResponseParser): + """Subclass of HttpResponseParser to make it patchable.""" + + with mock.patch( + "aiohttp.client_proto.HttpResponseParser", PatchableHttpResponseParser + ): + transport = mock.create_autospec( + asyncio.Transport, spec_set=True, instance=True + ) + proto.connection_made(transport) + proto.set_response_params(read_until_eof=True) + # Prime the parser so feed_data has been called once with valid data. + proto.data_received(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nab") + transport.close.reset_mock() + + with mock.patch.object( + proto._parser, "feed_data", side_effect=asyncio.CancelledError + ): + with pytest.raises(asyncio.CancelledError): + proto.data_received(b"more") + + assert transport.close.called + + async def test_client_protocol_readuntil_eof(loop: asyncio.AbstractEventLoop) -> None: proto = ResponseHandler(loop=loop) transport = mock.Mock() From 377cdb16b715ce27d7ac09a33e670bd8a04a9029 Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Thu, 4 Jun 2026 22:47:31 +0100 Subject: [PATCH 18/49] Close transport on BaseException in ResponseHandler.data_received (#12798) (#12813) (cherry picked from commit 24cb57bb8755ac795224db2e24d5baf6c4735812) --- CHANGES/12795.bugfix.rst | 1 + aiohttp/client_proto.py | 4 +++- tests/test_client_proto.py | 29 +++++++++++++++++++++++++++++ 3 files changed, 33 insertions(+), 1 deletion(-) create mode 100644 CHANGES/12795.bugfix.rst diff --git a/CHANGES/12795.bugfix.rst b/CHANGES/12795.bugfix.rst new file mode 100644 index 00000000000..d08ea779287 --- /dev/null +++ b/CHANGES/12795.bugfix.rst @@ -0,0 +1 @@ +Fixed ``CancelledError`` not closing a connection -- by :user:`aiolibsbot`. diff --git a/aiohttp/client_proto.py b/aiohttp/client_proto.py index e996d43eb2e..a0b8512af9b 100644 --- a/aiohttp/client_proto.py +++ b/aiohttp/client_proto.py @@ -322,12 +322,14 @@ def data_received(self, data: bytes) -> None: # parse http messages try: messages, upgraded, tail = self._parser.feed_data(data) - except Exception as underlying_exc: + except BaseException as underlying_exc: if self.transport is not None: # connection.release() could be called BEFORE # data_received(), the transport is already # closed in this case self.transport.close() + if not isinstance(underlying_exc, Exception): + raise # should_close is True after the call if isinstance(underlying_exc, HttpProcessingError): exc = HttpProcessingError( diff --git a/tests/test_client_proto.py b/tests/test_client_proto.py index ed6e602b0d5..b4cd15b00f8 100644 --- a/tests/test_client_proto.py +++ b/tests/test_client_proto.py @@ -1,6 +1,7 @@ import asyncio from unittest import mock +import pytest from yarl import URL from aiohttp import http @@ -164,6 +165,34 @@ class PatchableHttpResponseParser(http.HttpResponseParser): assert isinstance(proto.exception(), http.HttpProcessingError) +async def test_base_exception_during_data_received_closes_transport() -> None: + loop = asyncio.get_running_loop() + proto = ResponseHandler(loop=loop) + + class PatchableHttpResponseParser(http.HttpResponseParser): + """Subclass of HttpResponseParser to make it patchable.""" + + with mock.patch( + "aiohttp.client_proto.HttpResponseParser", PatchableHttpResponseParser + ): + transport = mock.create_autospec( + asyncio.Transport, spec_set=True, instance=True + ) + proto.connection_made(transport) + proto.set_response_params(read_until_eof=True) + # Prime the parser so feed_data has been called once with valid data. + proto.data_received(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nab") + transport.close.reset_mock() + + with mock.patch.object( + proto._parser, "feed_data", side_effect=asyncio.CancelledError + ): + with pytest.raises(asyncio.CancelledError): + proto.data_received(b"more") + + assert transport.close.called + + async def test_client_protocol_readuntil_eof(loop: asyncio.AbstractEventLoop) -> None: proto = ResponseHandler(loop=loop) transport = mock.Mock() From 451a8de87e5382b18c8bab8f1c3a2ab9e986da28 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Fri, 5 Jun 2026 18:30:32 +0000 Subject: [PATCH 19/49] [PR #12809/5771a85f backport][3.14] Revert "Move pip-tools into pyproject.toml" (#12810) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: 🇺🇦 Sviatoslav Sydorenko (Святослав Сидоренко) --- .pip-tools.toml | 5 +++++ pyproject.toml | 7 ------- 2 files changed, 5 insertions(+), 7 deletions(-) create mode 100644 .pip-tools.toml diff --git a/.pip-tools.toml b/.pip-tools.toml new file mode 100644 index 00000000000..86969d6eda2 --- /dev/null +++ b/.pip-tools.toml @@ -0,0 +1,5 @@ +[pip-tools] +allow-unsafe = false +resolver = "backtracking" +strip-extras = true +unsafe-package = "aiohttp" diff --git a/pyproject.toml b/pyproject.toml index 4011ba49ce2..bf490bf7cdc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -175,13 +175,6 @@ skip = "pp*" # iOS currently does not support build[uv] build-frontend = "build" -[tool.pip-tools] -# Dependabot won't pick up .pip-tools.toml, so must be defined here. -allow-unsafe = false -resolver = "backtracking" -strip-extras = true -unsafe-package = ["aiohttp"] - [tool.codespell] skip = '.git,*.pdf,*.svg,Makefile,CONTRIBUTORS.txt,venvs,_build' ignore-words-list = 'te,ue' From f09ae93d6ad1e1c236de059b1905ccb58b4fb87f Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sat, 6 Jun 2026 00:05:07 +0100 Subject: [PATCH 20/49] [PR #12817/69344c6e backport][3.15] Improve websocket checks (#12819) **This is a backport of PR #12817 as merged into master (69344c6efa3e5dd80b1c88079fa06d4e902a3b83).** Co-authored-by: Sam Bull --- CHANGES/12817.bugfix.rst | 1 + aiohttp/_websocket/reader_py.py | 42 +++++++++++++------- tests/test_websocket_parser.py | 68 +++++++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+), 13 deletions(-) create mode 100644 CHANGES/12817.bugfix.rst diff --git a/CHANGES/12817.bugfix.rst b/CHANGES/12817.bugfix.rst new file mode 100644 index 00000000000..c8a35e309a0 --- /dev/null +++ b/CHANGES/12817.bugfix.rst @@ -0,0 +1 @@ +Tightened up some websocket parser checks -- by :user:`Dreamsorcerer`. diff --git a/aiohttp/_websocket/reader_py.py b/aiohttp/_websocket/reader_py.py index ca15c3b2b5c..784fb08b0ee 100644 --- a/aiohttp/_websocket/reader_py.py +++ b/aiohttp/_websocket/reader_py.py @@ -208,12 +208,6 @@ def _handle_frame( if opcode != OP_CODE_CONTINUATION: self._opcode = opcode self._partial += payload - if self._max_msg_size and len(self._partial) >= self._max_msg_size: - raise WebSocketError( - WSCloseCode.MESSAGE_TOO_BIG, - f"Message size {len(self._partial)} " - f"exceeds limit {self._max_msg_size}", - ) return has_partial = bool(self._partial) @@ -236,13 +230,6 @@ def _handle_frame( else: assembled_payload = payload - if self._max_msg_size and len(assembled_payload) >= self._max_msg_size: - raise WebSocketError( - WSCloseCode.MESSAGE_TOO_BIG, - f"Message size {len(assembled_payload)} " - f"exceeds limit {self._max_msg_size}", - ) - # Decompress process must to be done after all packets # received. if compressed: @@ -376,6 +363,19 @@ def _feed_data(self, data: bytes) -> None: "Received frame with non-zero reserved bits", ) + if opcode not in { + OP_CODE_CONTINUATION, + OP_CODE_TEXT, + OP_CODE_BINARY, + OP_CODE_CLOSE, + OP_CODE_PING, + OP_CODE_PONG, + }: + raise WebSocketError( + WSCloseCode.PROTOCOL_ERROR, + f"Unexpected opcode={opcode!r}", + ) + if opcode > 0x7 and fin == 0: raise WebSocketError( WSCloseCode.PROTOCOL_ERROR, @@ -428,6 +428,22 @@ def _feed_data(self, data: bytes) -> None: else: self._payload_bytes_to_read = len_flag + # Reject oversized data frames before buffering any payload + # bytes. Control frames are capped at 125 bytes (checked in + # READ_HEADER) so only text/binary/continuation need this. + if self._max_msg_size and self._frame_opcode in { + OP_CODE_TEXT, + OP_CODE_BINARY, + OP_CODE_CONTINUATION, + }: + projected_size = self._payload_bytes_to_read + len(self._partial) + if projected_size >= self._max_msg_size: + raise WebSocketError( + WSCloseCode.MESSAGE_TOO_BIG, + f"Message size {projected_size} " + f"exceeds limit {self._max_msg_size}", + ) + self._state = READ_PAYLOAD_MASK if self._has_mask else READ_PAYLOAD # read payload mask diff --git a/tests/test_websocket_parser.py b/tests/test_websocket_parser.py index 01e786787f7..0e5890c72b4 100644 --- a/tests/test_websocket_parser.py +++ b/tests/test_websocket_parser.py @@ -643,6 +643,74 @@ def test_compressed_msg_too_large(out) -> None: assert ctx.value.code == WSCloseCode.MESSAGE_TOO_BIG +@pytest.mark.parametrize("fin", (0x80, 0x00), ids=("fin", "non-fin")) +def test_msg_too_large_at_header(out: WebSocketDataQueue, fin: int) -> None: + max_msg_size = 256 + parser = WebSocketReader(out, max_msg_size, compress=False) + + # Header alone: TEXT, 64-bit length, declares 1 MiB of payload. + header = PACK_LEN3(fin | WSMsgType.TEXT, 127, 1024 * 1024) + with pytest.raises( + WebSocketError, match=r"^Message size 1048576 exceeds limit 256$" + ) as ctx: + parser._feed_data(header) + assert ctx.value.code == WSCloseCode.MESSAGE_TOO_BIG + + +def test_msg_too_large_across_fragments(out: WebSocketDataQueue) -> None: + # Individual fragments fit under max_msg_size but accumulate past it. + max_msg_size = 256 + parser = WebSocketReader(out, max_msg_size, compress=False) + + first = build_frame(b"a" * 100, WSMsgType.TEXT, is_fin=False) + parser._feed_data(first) + middle = build_frame(b"b" * 100, WSMsgType.CONTINUATION, is_fin=False) + parser._feed_data(middle) + + # Third 100-byte fragment would push the accumulated total to 300. + last = build_frame(b"c" * 100, WSMsgType.CONTINUATION, is_fin=False) + with pytest.raises( + WebSocketError, match=r"^Message size 300 exceeds limit 256$" + ) as ctx: + parser._feed_data(last) + assert ctx.value.code == WSCloseCode.MESSAGE_TOO_BIG + + +def test_msg_too_large_text_after_non_fin_text(out: WebSocketDataQueue) -> None: + # Protocol-violating sequence: a fresh TEXT arrives while a fragmented + # message is still open. + max_msg_size = 256 + parser = WebSocketReader(out, max_msg_size, compress=False) + + first = build_frame(b"a" * 200, WSMsgType.TEXT, is_fin=False) + parser._feed_data(first) + + # Second TEXT header alone announces 100 bytes; 100 + 200 partial = 300. + second_header = PACK_LEN1(WSMsgType.TEXT, 100) + with pytest.raises( + WebSocketError, match=r"^Message size 300 exceeds limit 256$" + ) as ctx: + parser._feed_data(second_header) + assert ctx.value.code == WSCloseCode.MESSAGE_TOO_BIG + + +@pytest.mark.parametrize( + "opcode", + (0x3, 0x4, 0x5, 0x6, 0x7, 0xB, 0xC, 0xD, 0xE, 0xF), + ids=lambda v: f"0x{v:x}", +) +def test_reserved_opcode_rejected_at_header( + out: WebSocketDataQueue, opcode: int +) -> None: + # RFC 6455 reserves opcodes 0x3-0x7 (non-control) and 0xB-0xF (control). + parser = WebSocketReader(out, max_msg_size=256, compress=False) + + header = PACK_LEN3(0x80 | opcode, 127, 1024 * 1024) + with pytest.raises(WebSocketError, match=rf"^Unexpected opcode={opcode}$") as ctx: + parser._feed_data(header) + assert ctx.value.code == WSCloseCode.PROTOCOL_ERROR + + class TestWebSocketError: def test_ctor(self) -> None: err = WebSocketError(WSCloseCode.PROTOCOL_ERROR, "Something invalid") From 14b6ee851fb16ec199acb950de0c82d476799e7d Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sat, 6 Jun 2026 00:05:19 +0100 Subject: [PATCH 21/49] [PR #12817/69344c6e backport][3.14] Improve websocket checks (#12818) **This is a backport of PR #12817 as merged into master (69344c6efa3e5dd80b1c88079fa06d4e902a3b83).** Co-authored-by: Sam Bull --- CHANGES/12817.bugfix.rst | 1 + aiohttp/_websocket/reader_py.py | 42 +++++++++++++------- tests/test_websocket_parser.py | 68 +++++++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+), 13 deletions(-) create mode 100644 CHANGES/12817.bugfix.rst diff --git a/CHANGES/12817.bugfix.rst b/CHANGES/12817.bugfix.rst new file mode 100644 index 00000000000..c8a35e309a0 --- /dev/null +++ b/CHANGES/12817.bugfix.rst @@ -0,0 +1 @@ +Tightened up some websocket parser checks -- by :user:`Dreamsorcerer`. diff --git a/aiohttp/_websocket/reader_py.py b/aiohttp/_websocket/reader_py.py index ca15c3b2b5c..784fb08b0ee 100644 --- a/aiohttp/_websocket/reader_py.py +++ b/aiohttp/_websocket/reader_py.py @@ -208,12 +208,6 @@ def _handle_frame( if opcode != OP_CODE_CONTINUATION: self._opcode = opcode self._partial += payload - if self._max_msg_size and len(self._partial) >= self._max_msg_size: - raise WebSocketError( - WSCloseCode.MESSAGE_TOO_BIG, - f"Message size {len(self._partial)} " - f"exceeds limit {self._max_msg_size}", - ) return has_partial = bool(self._partial) @@ -236,13 +230,6 @@ def _handle_frame( else: assembled_payload = payload - if self._max_msg_size and len(assembled_payload) >= self._max_msg_size: - raise WebSocketError( - WSCloseCode.MESSAGE_TOO_BIG, - f"Message size {len(assembled_payload)} " - f"exceeds limit {self._max_msg_size}", - ) - # Decompress process must to be done after all packets # received. if compressed: @@ -376,6 +363,19 @@ def _feed_data(self, data: bytes) -> None: "Received frame with non-zero reserved bits", ) + if opcode not in { + OP_CODE_CONTINUATION, + OP_CODE_TEXT, + OP_CODE_BINARY, + OP_CODE_CLOSE, + OP_CODE_PING, + OP_CODE_PONG, + }: + raise WebSocketError( + WSCloseCode.PROTOCOL_ERROR, + f"Unexpected opcode={opcode!r}", + ) + if opcode > 0x7 and fin == 0: raise WebSocketError( WSCloseCode.PROTOCOL_ERROR, @@ -428,6 +428,22 @@ def _feed_data(self, data: bytes) -> None: else: self._payload_bytes_to_read = len_flag + # Reject oversized data frames before buffering any payload + # bytes. Control frames are capped at 125 bytes (checked in + # READ_HEADER) so only text/binary/continuation need this. + if self._max_msg_size and self._frame_opcode in { + OP_CODE_TEXT, + OP_CODE_BINARY, + OP_CODE_CONTINUATION, + }: + projected_size = self._payload_bytes_to_read + len(self._partial) + if projected_size >= self._max_msg_size: + raise WebSocketError( + WSCloseCode.MESSAGE_TOO_BIG, + f"Message size {projected_size} " + f"exceeds limit {self._max_msg_size}", + ) + self._state = READ_PAYLOAD_MASK if self._has_mask else READ_PAYLOAD # read payload mask diff --git a/tests/test_websocket_parser.py b/tests/test_websocket_parser.py index 01e786787f7..0e5890c72b4 100644 --- a/tests/test_websocket_parser.py +++ b/tests/test_websocket_parser.py @@ -643,6 +643,74 @@ def test_compressed_msg_too_large(out) -> None: assert ctx.value.code == WSCloseCode.MESSAGE_TOO_BIG +@pytest.mark.parametrize("fin", (0x80, 0x00), ids=("fin", "non-fin")) +def test_msg_too_large_at_header(out: WebSocketDataQueue, fin: int) -> None: + max_msg_size = 256 + parser = WebSocketReader(out, max_msg_size, compress=False) + + # Header alone: TEXT, 64-bit length, declares 1 MiB of payload. + header = PACK_LEN3(fin | WSMsgType.TEXT, 127, 1024 * 1024) + with pytest.raises( + WebSocketError, match=r"^Message size 1048576 exceeds limit 256$" + ) as ctx: + parser._feed_data(header) + assert ctx.value.code == WSCloseCode.MESSAGE_TOO_BIG + + +def test_msg_too_large_across_fragments(out: WebSocketDataQueue) -> None: + # Individual fragments fit under max_msg_size but accumulate past it. + max_msg_size = 256 + parser = WebSocketReader(out, max_msg_size, compress=False) + + first = build_frame(b"a" * 100, WSMsgType.TEXT, is_fin=False) + parser._feed_data(first) + middle = build_frame(b"b" * 100, WSMsgType.CONTINUATION, is_fin=False) + parser._feed_data(middle) + + # Third 100-byte fragment would push the accumulated total to 300. + last = build_frame(b"c" * 100, WSMsgType.CONTINUATION, is_fin=False) + with pytest.raises( + WebSocketError, match=r"^Message size 300 exceeds limit 256$" + ) as ctx: + parser._feed_data(last) + assert ctx.value.code == WSCloseCode.MESSAGE_TOO_BIG + + +def test_msg_too_large_text_after_non_fin_text(out: WebSocketDataQueue) -> None: + # Protocol-violating sequence: a fresh TEXT arrives while a fragmented + # message is still open. + max_msg_size = 256 + parser = WebSocketReader(out, max_msg_size, compress=False) + + first = build_frame(b"a" * 200, WSMsgType.TEXT, is_fin=False) + parser._feed_data(first) + + # Second TEXT header alone announces 100 bytes; 100 + 200 partial = 300. + second_header = PACK_LEN1(WSMsgType.TEXT, 100) + with pytest.raises( + WebSocketError, match=r"^Message size 300 exceeds limit 256$" + ) as ctx: + parser._feed_data(second_header) + assert ctx.value.code == WSCloseCode.MESSAGE_TOO_BIG + + +@pytest.mark.parametrize( + "opcode", + (0x3, 0x4, 0x5, 0x6, 0x7, 0xB, 0xC, 0xD, 0xE, 0xF), + ids=lambda v: f"0x{v:x}", +) +def test_reserved_opcode_rejected_at_header( + out: WebSocketDataQueue, opcode: int +) -> None: + # RFC 6455 reserves opcodes 0x3-0x7 (non-control) and 0xB-0xF (control). + parser = WebSocketReader(out, max_msg_size=256, compress=False) + + header = PACK_LEN3(0x80 | opcode, 127, 1024 * 1024) + with pytest.raises(WebSocketError, match=rf"^Unexpected opcode={opcode}$") as ctx: + parser._feed_data(header) + assert ctx.value.code == WSCloseCode.PROTOCOL_ERROR + + class TestWebSocketError: def test_ctor(self) -> None: err = WebSocketError(WSCloseCode.PROTOCOL_ERROR, "Something invalid") From 0d174914280f89d4cd56979c26c38882d9c4b56a Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 00:28:36 -0500 Subject: [PATCH 22/49] [PR #12832/5e898429 backport][3.15] Enforce max_line_size on complete chunk-size lines in pure-Python parser (#12852) Co-authored-by: J. Nick Koston --- CHANGES/12832.bugfix.rst | 1 + aiohttp/http_parser.py | 4 +++ tests/test_http_parser.py | 53 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+) create mode 100644 CHANGES/12832.bugfix.rst diff --git a/CHANGES/12832.bugfix.rst b/CHANGES/12832.bugfix.rst new file mode 100644 index 00000000000..00562fecd61 --- /dev/null +++ b/CHANGES/12832.bugfix.rst @@ -0,0 +1 @@ +Fixed the pure-Python HTTP parser not enforcing ``max_line_size`` on a chunk-size line when the whole line arrived in a single read; the limit was only applied to chunk-size metadata split across reads. The complete-line case is now checked too, matching the split-line behavior -- by :user:`bdraco`. diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py index 95a1a3aa8aa..5d48dedaa6c 100644 --- a/aiohttp/http_parser.py +++ b/aiohttp/http_parser.py @@ -945,6 +945,10 @@ def feed_data( if self._chunk == ChunkState.PARSE_CHUNKED_SIZE: pos = chunk.find(SEP) if pos >= 0: + # Only chunk-size lines reach here; trailers enforce + # _max_field_size separately in PARSE_TRAILERS below. + if pos > self._max_line_size: + raise LineTooLong(chunk[:100] + b"...", self._max_line_size) i = chunk.find(CHUNK_EXT, 0, pos) if i >= 0: size_b = chunk[:i] # strip chunk-extensions diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index 935cf65a493..b9125fa4078 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -2280,6 +2280,59 @@ async def test_parse_chunked_payload_size_error( p.feed_data(b"blah\r\n") assert isinstance(out.exception(), http_exceptions.TransferEncodingError) + async def test_chunked_chunk_size_line_too_long( + self, protocol: BaseProtocol + ) -> None: + """A complete oversized chunk-size line is rejected with LineTooLong.""" + out = aiohttp.StreamReader(protocol, 2**16, loop=asyncio.get_running_loop()) + p = HttpPayloadParser( + out, chunked=True, headers_parser=HeadersParser(), max_line_size=32 + ) + size_line = b"1;" + b"a" * 4096 + b"\r\n" + with pytest.raises(http_exceptions.LineTooLong): + p.feed_data(size_line) + + async def test_chunked_chunk_size_line_within_limit( + self, protocol: BaseProtocol + ) -> None: + """A small chunk-size line still parses when max_line_size is low.""" + out = aiohttp.StreamReader(protocol, 2**16, loop=asyncio.get_running_loop()) + p = HttpPayloadParser( + out, chunked=True, headers_parser=HeadersParser(), max_line_size=32 + ) + p.feed_data(b"1\r\nx\r\n0\r\n\r\n") + assert out.is_eof() + assert b"x" == b"".join(out._buffer) + + async def test_chunked_chunk_size_line_at_limit( + self, protocol: BaseProtocol + ) -> None: + """A chunk-size line of exactly max_line_size bytes is accepted (>, not >=).""" + out = aiohttp.StreamReader(protocol, 2**16, loop=asyncio.get_running_loop()) + p = HttpPayloadParser( + out, chunked=True, headers_parser=HeadersParser(), max_line_size=32 + ) + # "1;" + 30 * "a" is exactly 32 bytes before the CRLF. + size_line = b"1;" + b"a" * 30 + assert len(size_line) == 32 + p.feed_data(size_line + b"\r\nx\r\n0\r\n\r\n") + assert out.is_eof() + assert b"x" == b"".join(out._buffer) + + async def test_chunked_chunk_size_line_one_over_limit( + self, protocol: BaseProtocol + ) -> None: + """A chunk-size line one byte over max_line_size is rejected.""" + out = aiohttp.StreamReader(protocol, 2**16, loop=asyncio.get_running_loop()) + p = HttpPayloadParser( + out, chunked=True, headers_parser=HeadersParser(), max_line_size=32 + ) + # "1;" + 31 * "a" is 33 bytes before the CRLF. + size_line = b"1;" + b"a" * 31 + assert len(size_line) == 33 + with pytest.raises(http_exceptions.LineTooLong): + p.feed_data(size_line + b"\r\nx\r\n0\r\n\r\n") + async def test_parse_chunked_payload_size_data_mismatch( self, protocol: BaseProtocol ) -> None: From b411319e641b347927f6a65b0d55b3f7b04b80b6 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:28:49 -0500 Subject: [PATCH 23/49] [PR #12835/1e94b3e8 backport][3.15] Tls server hostname pool key (#12848) --- CHANGES/12835.bugfix.rst | 1 + aiohttp/client_reqrep.py | 2 ++ tests/test_client_functional.py | 31 +++++++++++++++++++++++++++++++ tests/test_client_request.py | 16 ++++++++++++++++ 4 files changed, 50 insertions(+) create mode 100644 CHANGES/12835.bugfix.rst diff --git a/CHANGES/12835.bugfix.rst b/CHANGES/12835.bugfix.rst new file mode 100644 index 00000000000..84a8ae00677 --- /dev/null +++ b/CHANGES/12835.bugfix.rst @@ -0,0 +1 @@ +Included the per-request ``server_hostname`` override in the :class:`~aiohttp.TCPConnector` connection pool key, so a pooled TLS connection is no longer reused for a request that sets ``server_hostname`` to a different value -- by :user:`bdraco`. diff --git a/aiohttp/client_reqrep.py b/aiohttp/client_reqrep.py index 4aeae08ac19..c77213e3ed2 100644 --- a/aiohttp/client_reqrep.py +++ b/aiohttp/client_reqrep.py @@ -244,6 +244,7 @@ class ConnectionKey(NamedTuple): proxy: URL | None proxy_auth: BasicAuth | None proxy_headers_hash: int | None # hash(CIMultiDict) + server_hostname: str | None = None def _is_expected_content_type( @@ -982,6 +983,7 @@ def connection_key(self) -> ConnectionKey: self.proxy, self.proxy_auth, h, + self.server_hostname, ), ) diff --git a/tests/test_client_functional.py b/tests/test_client_functional.py index aa3a6925c2f..6bd57bee612 100644 --- a/tests/test_client_functional.py +++ b/tests/test_client_functional.py @@ -722,6 +722,37 @@ async def handler(request): assert txt == "Test message" +async def test_server_hostname_override_not_reused( + aiohttp_server: AiohttpServer, +) -> None: + """A pooled TLS connection must not be reused for a different server_hostname.""" + trustme = pytest.importorskip("trustme") + + ca = trustme.CA() + cert = ca.issue_cert("first.example") + server_ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + cert.configure_cert(server_ctx) + client_ctx = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH) + ca.configure_trust(client_ctx) + + async def handler(request: web.Request) -> web.Response: + return web.Response(text="ok") + + app = web.Application() + app.router.add_route("GET", "/", handler) + server = await aiohttp_server(app, ssl=server_ctx) + url = server.make_url("/") + + connector = aiohttp.TCPConnector(ssl=client_ctx, limit=1, limit_per_host=1) + async with aiohttp.ClientSession(connector=connector) as session: + async with session.get(url, server_hostname="first.example") as resp: + assert resp.status == 200 + await resp.read() + + with pytest.raises(aiohttp.ClientConnectorCertificateError): + await session.get(url, server_hostname="second.example") + + @pytest.mark.skipif( sys.version_info < (3, 11), reason="ssl_shutdown_timeout requires Python 3.11+" ) diff --git a/tests/test_client_request.py b/tests/test_client_request.py index 66240258620..90ffcecb5d8 100644 --- a/tests/test_client_request.py +++ b/tests/test_client_request.py @@ -1614,6 +1614,22 @@ async def test_connection_key_without_proxy() -> None: await req.close() +async def test_connection_key_includes_server_hostname( + make_request: _RequestMaker, +) -> None: + """A server_hostname override must be part of the connection reuse key.""" + url = URL("https://127.0.0.1:8443/") + none_req = make_request("GET", url) + first = make_request("GET", url, server_hostname="first.example") + first_again = make_request("GET", url, server_hostname="first.example") + second = make_request("GET", url, server_hostname="second.example") + + assert first.connection_key.server_hostname == "first.example" + assert first.connection_key != none_req.connection_key + assert first.connection_key != second.connection_key + assert first.connection_key == first_again.connection_key + + def test_request_info_back_compat() -> None: """Test RequestInfo can be created without real_url.""" url = URL("http://example.com") From 758bacc2a2eecdbce1f03916e045020735431c18 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 00:28:57 -0500 Subject: [PATCH 24/49] [PR #12832/5e898429 backport][3.14] Enforce max_line_size on complete chunk-size lines in pure-Python parser (#12851) Co-authored-by: J. Nick Koston --- CHANGES/12832.bugfix.rst | 1 + aiohttp/http_parser.py | 4 +++ tests/test_http_parser.py | 53 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+) create mode 100644 CHANGES/12832.bugfix.rst diff --git a/CHANGES/12832.bugfix.rst b/CHANGES/12832.bugfix.rst new file mode 100644 index 00000000000..00562fecd61 --- /dev/null +++ b/CHANGES/12832.bugfix.rst @@ -0,0 +1 @@ +Fixed the pure-Python HTTP parser not enforcing ``max_line_size`` on a chunk-size line when the whole line arrived in a single read; the limit was only applied to chunk-size metadata split across reads. The complete-line case is now checked too, matching the split-line behavior -- by :user:`bdraco`. diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py index 95a1a3aa8aa..5d48dedaa6c 100644 --- a/aiohttp/http_parser.py +++ b/aiohttp/http_parser.py @@ -945,6 +945,10 @@ def feed_data( if self._chunk == ChunkState.PARSE_CHUNKED_SIZE: pos = chunk.find(SEP) if pos >= 0: + # Only chunk-size lines reach here; trailers enforce + # _max_field_size separately in PARSE_TRAILERS below. + if pos > self._max_line_size: + raise LineTooLong(chunk[:100] + b"...", self._max_line_size) i = chunk.find(CHUNK_EXT, 0, pos) if i >= 0: size_b = chunk[:i] # strip chunk-extensions diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index 935cf65a493..b9125fa4078 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -2280,6 +2280,59 @@ async def test_parse_chunked_payload_size_error( p.feed_data(b"blah\r\n") assert isinstance(out.exception(), http_exceptions.TransferEncodingError) + async def test_chunked_chunk_size_line_too_long( + self, protocol: BaseProtocol + ) -> None: + """A complete oversized chunk-size line is rejected with LineTooLong.""" + out = aiohttp.StreamReader(protocol, 2**16, loop=asyncio.get_running_loop()) + p = HttpPayloadParser( + out, chunked=True, headers_parser=HeadersParser(), max_line_size=32 + ) + size_line = b"1;" + b"a" * 4096 + b"\r\n" + with pytest.raises(http_exceptions.LineTooLong): + p.feed_data(size_line) + + async def test_chunked_chunk_size_line_within_limit( + self, protocol: BaseProtocol + ) -> None: + """A small chunk-size line still parses when max_line_size is low.""" + out = aiohttp.StreamReader(protocol, 2**16, loop=asyncio.get_running_loop()) + p = HttpPayloadParser( + out, chunked=True, headers_parser=HeadersParser(), max_line_size=32 + ) + p.feed_data(b"1\r\nx\r\n0\r\n\r\n") + assert out.is_eof() + assert b"x" == b"".join(out._buffer) + + async def test_chunked_chunk_size_line_at_limit( + self, protocol: BaseProtocol + ) -> None: + """A chunk-size line of exactly max_line_size bytes is accepted (>, not >=).""" + out = aiohttp.StreamReader(protocol, 2**16, loop=asyncio.get_running_loop()) + p = HttpPayloadParser( + out, chunked=True, headers_parser=HeadersParser(), max_line_size=32 + ) + # "1;" + 30 * "a" is exactly 32 bytes before the CRLF. + size_line = b"1;" + b"a" * 30 + assert len(size_line) == 32 + p.feed_data(size_line + b"\r\nx\r\n0\r\n\r\n") + assert out.is_eof() + assert b"x" == b"".join(out._buffer) + + async def test_chunked_chunk_size_line_one_over_limit( + self, protocol: BaseProtocol + ) -> None: + """A chunk-size line one byte over max_line_size is rejected.""" + out = aiohttp.StreamReader(protocol, 2**16, loop=asyncio.get_running_loop()) + p = HttpPayloadParser( + out, chunked=True, headers_parser=HeadersParser(), max_line_size=32 + ) + # "1;" + 31 * "a" is 33 bytes before the CRLF. + size_line = b"1;" + b"a" * 31 + assert len(size_line) == 33 + with pytest.raises(http_exceptions.LineTooLong): + p.feed_data(size_line + b"\r\nx\r\n0\r\n\r\n") + async def test_parse_chunked_payload_size_data_mismatch( self, protocol: BaseProtocol ) -> None: From bb248eb2fffb25b7d29f72c2e47afe528900fe20 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:29:51 -0500 Subject: [PATCH 25/49] [PR #12827/ccf218ab backport][3.15] Numeric ipv4 resolver bypass (#12850) --- CHANGES/12827.bugfix.rst | 1 + aiohttp/connector.py | 7 ++++ aiohttp/helpers.py | 22 ++++++++++ tests/test_connector.py | 30 ++++++++++++++ tests/test_helpers.py | 89 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 149 insertions(+) create mode 100644 CHANGES/12827.bugfix.rst diff --git a/CHANGES/12827.bugfix.rst b/CHANGES/12827.bugfix.rst new file mode 100644 index 00000000000..9442867d363 --- /dev/null +++ b/CHANGES/12827.bugfix.rst @@ -0,0 +1 @@ +Changed :class:`~aiohttp.TCPConnector` to reject legacy non-canonical numeric IPv4 host forms such as ``2130706433``, ``017700000001`` and ``127.1`` with :exc:`~aiohttp.InvalidUrlClientError`; only canonical dotted-quad IPv4 literals are now treated as IP address literals, while every other host is sent through the configured resolver -- by :user:`bdraco`. diff --git a/aiohttp/connector.py b/aiohttp/connector.py index b206f2e989a..b7aa7b7647a 100644 --- a/aiohttp/connector.py +++ b/aiohttp/connector.py @@ -27,6 +27,7 @@ ClientConnectorSSLError, ClientHttpProxyError, ClientProxyConnectionError, + InvalidUrlClientError, ServerFingerprintMismatch, UnixClientConnectorError, cert_errors, @@ -37,6 +38,7 @@ from .helpers import ( _SENTINEL, ceil_timeout, + is_canonical_ipv4_address, is_ip_address, noop, sentinel, @@ -1092,6 +1094,11 @@ async def _resolve_host( ) -> list[ResolveResult]: """Resolve host and return list of addresses.""" if is_ip_address(host): + # Reject legacy numeric IPv4 forms (e.g. 2130706433, 127.1) that + # socket would map onto an address, slipping past a connector-level + # policy that only sees the raw host. + if ":" not in host and not is_canonical_ipv4_address(host): + raise InvalidUrlClientError(host, "is not a canonical IPv4 address") return [ { "hostname": host, diff --git a/aiohttp/helpers.py b/aiohttp/helpers.py index 5988aab4aa8..469c99dd63c 100644 --- a/aiohttp/helpers.py +++ b/aiohttp/helpers.py @@ -511,6 +511,28 @@ def is_ip_address(host: str | None) -> bool: return ":" in host or host.replace(".", "").isdigit() +def is_canonical_ipv4_address(host: str) -> bool: + """Check if host is a canonical dotted-quad IPv4 address. + + Rejects the legacy numeric forms that ``socket`` still accepts and + maps onto an address, e.g. ``2130706433``, ``017700000001``, ``127.1``. + """ + parts = host.split(".") + if len(parts) != 4: + return False + for part in parts: + # Each octet must be 1-3 ASCII digits; reject unicode digits + # (which ``str.isdigit`` accepts but ``int`` may not), octal + # leading zeros, and values above 255. + if not (1 <= len(part) <= 3) or not part.isascii() or not part.isdigit(): + return False + if part[0] == "0" and len(part) != 1: + return False + if int(part) > 255: + return False + return True + + _cached_current_datetime: int | None = None _cached_formatted_datetime = "" diff --git a/tests/test_connector.py b/tests/test_connector.py index 27a7cc43ff6..75bd1461ba1 100644 --- a/tests/test_connector.py +++ b/tests/test_connector.py @@ -26,6 +26,7 @@ from aiohttp import client, connector as connector_module, hdrs, web from aiohttp.abc import AbstractResolver from aiohttp.client import ClientRequest, ClientTimeout +from aiohttp.client_exceptions import InvalidUrlClientError from aiohttp.client_proto import ResponseHandler from aiohttp.client_reqrep import ConnectionKey from aiohttp.connector import ( @@ -1253,6 +1254,35 @@ async def test_tcp_connector_resolve_host(loop: asyncio.AbstractEventLoop) -> No await conn.close() +async def test_tcp_connector_rejects_non_canonical_ipv4_alias() -> None: + """Legacy numeric IPv4 aliases must not bypass the configured resolver.""" + calls: list[str] = [] + + class _RecordingResolver(AbstractResolver): + async def resolve( + self, + host: str, + port: int = 0, + family: socket.AddressFamily = socket.AF_INET, + ) -> list[ResolveResult]: + assert False + + async def close(self) -> None: + """Close the resolver.""" + + conn = aiohttp.TCPConnector(resolver=_RecordingResolver()) + for alias in ("2130706433", "017700000001", "127.1"): + with pytest.raises(InvalidUrlClientError, match="canonical IPv4"): + await conn._resolve_host(alias, 8080) + + # Resolver is never consulted, and a canonical IP still short-circuits it. + assert calls == [] + res = await conn._resolve_host("127.0.0.1", 8080) + assert res[0]["host"] == "127.0.0.1" + assert calls == [] + await conn.close() + + @pytest.fixture def dns_response(loop): async def coro(): diff --git a/tests/test_helpers.py b/tests/test_helpers.py index 3ddf79c7666..081b807cca4 100644 --- a/tests/test_helpers.py +++ b/tests/test_helpers.py @@ -2,6 +2,8 @@ import base64 import datetime import gc +import ipaddress +import itertools import sys import warnings import weakref @@ -319,6 +321,93 @@ def test_is_ip_address_invalid_type() -> None: helpers.is_ip_address(object()) +# ------------------------------- is_canonical_ipv4_address() --------------- + + +@pytest.mark.parametrize( + "host", + [ + "0.0.0.0", + "127.0.0.1", + "8.8.8.8", + "192.168.0.1", + "255.255.255.255", + ], +) +def test_is_canonical_ipv4_address_accepts_dotted_quad(host: str) -> None: + assert helpers.is_canonical_ipv4_address(host) + + +@pytest.mark.parametrize( + "host", + [ + "2130706433", # decimal integer form of 127.0.0.1 + "017700000001", # octal form of 127.0.0.1 + "127.1", # short-hand form of 127.0.0.1 + "127.0.1", # 3-part short-hand + "0177.0.0.1", # octal leading-zero octet + "01.2.3.4", # octal leading-zero octet + "256.0.0.1", # octet out of range + "999.0.0.1", # octet out of range + "1.2.3.4.5", # too many octets + "127.0.0.", # trailing dot / empty octet + "12³.0.0.1", # superscript digit (str.isdigit but not int) + "１２７.0.0.1", # full-width digits + "0xa.0.0.0", # hex octet + " 127.0.0.1", # leading whitespace + "127.0.0.1 ", # trailing whitespace + "example.com", # domain name + "", # empty + ], +) +def test_is_canonical_ipv4_address_rejects_non_canonical(host: str) -> None: + assert not helpers.is_canonical_ipv4_address(host) + + +def _ipaddress_accepts_ipv4(host: str) -> bool: + """Oracle: does the stdlib accept ``host`` as a canonical IPv4 address?""" + try: + ipaddress.IPv4Address(host) + except ipaddress.AddressValueError: + return False + return True + + +def test_is_canonical_ipv4_address_matches_stdlib() -> None: + """Prove equivalence with ``ipaddress.IPv4Address`` over a broad corpus. + + The helper is a fast hand-rolled substitute for the stdlib parser; this + exhaustively cross-checks the two agree on every combination of a set of + octet-like tokens covering the known edge cases (leading zeros, out of + range, empty, unicode digits, wrong octet count). + """ + tokens = [ + "0", + "1", + "9", + "10", + "99", + "255", + "256", + "999", + "00", + "01", + "0177", + "1234", + "", + "a", + "0x1", + "１", # full-width 1 + "1²", # trailing superscript + ] + for count in range(1, 5): + for parts in itertools.product(tokens, repeat=count): + host = ".".join(parts) + assert helpers.is_canonical_ipv4_address(host) == _ipaddress_accepts_ipv4( + host + ), host + + # ----------------------------------- TimeoutHandle ------------------- From 77633060a562387e0413bd646d20763175111bb4 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 00:30:17 -0500 Subject: [PATCH 26/49] [PR #12825/cb1d6a53 backport][3.15] Scope DigestAuthMiddleware credentials to the request origin (#12840) Co-authored-by: J. Nick Koston Co-authored-by: J. Nick Koston --- CHANGES/12825.bugfix.rst | 1 + aiohttp/client_middleware_digest_auth.py | 21 +++ docs/client_reference.rst | 14 ++ tests/test_client_middleware_digest_auth.py | 170 ++++++++++++++++++++ 4 files changed, 206 insertions(+) create mode 100644 CHANGES/12825.bugfix.rst diff --git a/CHANGES/12825.bugfix.rst b/CHANGES/12825.bugfix.rst new file mode 100644 index 00000000000..88d1bfe8c4c --- /dev/null +++ b/CHANGES/12825.bugfix.rst @@ -0,0 +1 @@ +Scoped :class:`~aiohttp.DigestAuthMiddleware` credentials to the origin of the first request it handles, so a redirect to a different origin no longer triggers a digest response computed from the configured credentials; a challenge from another origin is only answered when that origin falls within a protection space advertised by the anchor origin through the RFC 7616 ``domain`` directive -- by :user:`bdraco`. diff --git a/aiohttp/client_middleware_digest_auth.py b/aiohttp/client_middleware_digest_auth.py index 8151dea5154..6c3e37f7c00 100644 --- a/aiohttp/client_middleware_digest_auth.py +++ b/aiohttp/client_middleware_digest_auth.py @@ -162,6 +162,15 @@ class DigestAuthMiddleware: - Includes replay attack protection with client nonce count tracking - Supports preemptive authentication per RFC 7616 Section 3.6 + Origin scoping: + The credentials are scoped to the origin of the first request the + middleware handles. A request to a different origin is passed through + untouched, so it never receives a digest response computed from those + credentials, unless that origin falls within a protection space the + anchor origin advertised through the RFC 7616 ``domain`` directive. Make + the first request through the middleware against the intended origin, as + the anchor is pinned to it and not reset for the life of the instance. + Standards compliance: - RFC 7616: HTTP Digest Access Authentication (primary reference) - RFC 2617: HTTP Authentication (deprecated by RFC 7616) @@ -198,6 +207,8 @@ def __init__( self._preemptive: bool = preemptive # Set of URLs defining the protection space self._protection_space: list[str] = [] + # Origin the credentials are scoped to; set on the first request. + self._origin: URL | None = None async def _encode(self, method: str, url: URL, body: Payload | Literal[b""]) -> str: """ @@ -447,6 +458,16 @@ async def __call__( self, request: ClientRequest, handler: ClientHandlerType ) -> ClientResponse: """Run the digest auth middleware.""" + # Credentials are scoped to the first request's origin. Other origins + # pass through untouched unless a challenge from the anchor origin + # advertised them via RFC 7616 domain; mirrors aiohttp stripping + # Authorization on cross-origin redirects. + origin = request.url.origin() + if self._origin is None: + self._origin = origin + elif origin != self._origin and not self._in_protection_space(request.url): + return await handler(request) + response = None for retry_count in range(2): # Apply authorization header if: diff --git a/docs/client_reference.rst b/docs/client_reference.rst index 860e6fba119..734201128d6 100644 --- a/docs/client_reference.rst +++ b/docs/client_reference.rst @@ -2447,6 +2447,16 @@ Utilities The server may still respond with a 401 status and ``stale=true`` if the nonce has expired, in which case the middleware will automatically retry with the new nonce. + **Origin scoping** + + The credentials are scoped to the origin of the first request the middleware + handles. A request to a different origin is passed through untouched, so it + never receives a digest response computed from those credentials, unless that + origin falls within a protection space the anchor origin advertised through + the RFC 7616 ``domain`` directive. Make the first request through the + middleware against the intended origin, as the anchor is pinned to it and not + reset for the life of the instance. + To disable preemptive authentication and require a 401 challenge for every request, set ``preemptive=False``:: @@ -2472,6 +2482,10 @@ Utilities .. versionadded:: 3.12 .. versionchanged:: 3.12.8 Added ``preemptive`` parameter to enable/disable preemptive authentication. + .. versionchanged:: 3.14.1 + Credentials are scoped to the origin of the first request the middleware + handles; other origins are passed through untouched unless covered by an + RFC 7616 ``domain`` directive from the anchor origin. .. class:: CookieJar(*, unsafe=False, quote_cookie=True, treat_as_secure_origin = []) diff --git a/tests/test_client_middleware_digest_auth.py b/tests/test_client_middleware_digest_auth.py index 0d2d6ad3325..cdbedee980f 100644 --- a/tests/test_client_middleware_digest_auth.py +++ b/tests/test_client_middleware_digest_auth.py @@ -1198,6 +1198,176 @@ async def handler(request: Request) -> Response: ) # Second request - preemptive auth (entire origin) +async def test_does_not_answer_cross_origin_redirect_challenge( + aiohttp_server: AiohttpServer, +) -> None: + """A cross-origin redirect target must not receive a digest response. + + aiohttp strips the Authorization header on cross-origin redirects; the + digest middleware must not re-add one for the redirect target, otherwise + the configured credentials leak to an origin the caller never targeted. + """ + target_auth_headers: list[str | None] = [] + + async def target_handler(request: Request) -> Response: + auth_header = request.headers.get(hdrs.AUTHORIZATION) + target_auth_headers.append(auth_header) + assert auth_header is None + return Response( + status=401, + headers={ + hdrs.WWW_AUTHENTICATE: 'Digest realm="evil", nonce="cross-origin"' + }, + ) + + target_app = Application() + target_app.router.add_get("/", target_handler) + target_server = await aiohttp_server(target_app) + + async def source_handler(request: Request) -> Response: + return Response( + status=302, headers={hdrs.LOCATION: str(target_server.make_url("/"))} + ) + + source_app = Application() + source_app.router.add_get("/", source_handler) + source_server = await aiohttp_server(source_app) + + digest_auth = DigestAuthMiddleware("victim", "secret") + async with ( + ClientSession(middlewares=(digest_auth,)) as session, + session.get(source_server.make_url("/")) as response, + ): + await response.text() + + assert target_auth_headers == [None] + + +async def test_answers_same_origin_redirect_challenge( + aiohttp_server: AiohttpServer, +) -> None: + """A same-origin redirect that issues a challenge must still authenticate.""" + auth_headers: list[str | None] = [] + + async def handler(request: Request) -> Response: + if request.path == "/start": + return Response(status=302, headers={hdrs.LOCATION: "/protected"}) + auth_header = request.headers.get(hdrs.AUTHORIZATION) + auth_headers.append(auth_header) + if auth_header is None: + return Response( + status=401, + headers={hdrs.WWW_AUTHENTICATE: 'Digest realm="good", nonce="abc"'}, + ) + return Response(text="OK") + + app = Application() + app.router.add_get("/start", handler) + app.router.add_get("/protected", handler) + server = await aiohttp_server(app) + + digest_auth = DigestAuthMiddleware("user", "pass") + async with ( + ClientSession(middlewares=(digest_auth,)) as session, + session.get(server.make_url("/start")) as response, + ): + assert response.status == 200 + assert await response.text() == "OK" + + assert auth_headers[0] is None + assert auth_headers[1] is not None + assert auth_headers[1].startswith("Digest") + + +async def test_answers_cross_origin_within_domain_protection_space( + aiohttp_server: AiohttpServer, +) -> None: + """A different origin advertised via the ``domain`` directive is honored. + + RFC 7616 allows a challenge to define a protection space spanning other + servers through the ``domain`` directive. The anchor origin vouches for + those URIs, so preemptive auth to them is expected. + """ + other_auth_headers: list[str | None] = [] + + async def other_handler(request: Request) -> Response: + other_auth_headers.append(request.headers.get(hdrs.AUTHORIZATION)) + return Response(text="other") + + other_app = Application() + other_app.router.add_get("/", other_handler) + other_server = await aiohttp_server(other_app) + other_origin = str(other_server.make_url("/").origin()) + + async def anchor_handler(request: Request) -> Response: + if request.headers.get(hdrs.AUTHORIZATION) is None: + challenge = f'Digest realm="anchor", nonce="n1", domain="{other_origin}/"' + return Response(status=401, headers={hdrs.WWW_AUTHENTICATE: challenge}) + return Response(text="anchor") + + anchor_app = Application() + anchor_app.router.add_get("/", anchor_handler) + anchor_server = await aiohttp_server(anchor_app) + + digest_auth = DigestAuthMiddleware("user", "pass") + async with ClientSession(middlewares=(digest_auth,)) as session: + async with session.get(anchor_server.make_url("/")) as response: + assert response.status == 200 + async with session.get(other_server.make_url("/")) as response: + assert response.status == 200 + + assert other_auth_headers[0] is not None + assert other_auth_headers[0].startswith("Digest") + + +async def test_does_not_answer_cross_origin_challenge_without_redirect( + aiohttp_server: AiohttpServer, +) -> None: + """Origin scoping applies to any cross-origin request, not just redirects. + + After authenticating against the anchor origin, a direct request to a + different origin that issues its own challenge must not be answered with a + digest response computed from the configured credentials. + """ + other_auth_headers: list[str | None] = [] + + async def other_handler(request: Request) -> Response: + auth_header = request.headers.get(hdrs.AUTHORIZATION) + other_auth_headers.append(auth_header) + assert auth_header is None + return Response( + status=401, + headers={hdrs.WWW_AUTHENTICATE: 'Digest realm="evil", nonce="x"'}, + ) + + other_app = Application() + other_app.router.add_get("/", other_handler) + other_server = await aiohttp_server(other_app) + + async def anchor_handler(request: Request) -> Response: + if request.headers.get(hdrs.AUTHORIZATION) is None: + return Response( + status=401, + headers={hdrs.WWW_AUTHENTICATE: 'Digest realm="anchor", nonce="n1"'}, + ) + return Response(text="anchor") + + anchor_app = Application() + anchor_app.router.add_get("/", anchor_handler) + anchor_server = await aiohttp_server(anchor_app) + + digest_auth = DigestAuthMiddleware("user", "pass") + async with ClientSession(middlewares=(digest_auth,)) as session: + async with session.get(anchor_server.make_url("/")) as response: + assert response.status == 200 + async with session.get(other_server.make_url("/")) as response: + assert response.status == 401 + + # The other origin only ever saw the unauthenticated request; the + # middleware never answered its challenge. + assert other_auth_headers == [None] + + @pytest.mark.parametrize( ("status", "headers", "expected"), [ From 0ca2b6c28a25726527a8b60f25960262a91ed0e0 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:30:30 -0500 Subject: [PATCH 27/49] [PR #12835/1e94b3e8 backport][3.14] Tls server hostname pool key (#12847) --- CHANGES/12835.bugfix.rst | 1 + aiohttp/client_reqrep.py | 2 ++ tests/test_client_functional.py | 31 +++++++++++++++++++++++++++++++ tests/test_client_request.py | 16 ++++++++++++++++ 4 files changed, 50 insertions(+) create mode 100644 CHANGES/12835.bugfix.rst diff --git a/CHANGES/12835.bugfix.rst b/CHANGES/12835.bugfix.rst new file mode 100644 index 00000000000..84a8ae00677 --- /dev/null +++ b/CHANGES/12835.bugfix.rst @@ -0,0 +1 @@ +Included the per-request ``server_hostname`` override in the :class:`~aiohttp.TCPConnector` connection pool key, so a pooled TLS connection is no longer reused for a request that sets ``server_hostname`` to a different value -- by :user:`bdraco`. diff --git a/aiohttp/client_reqrep.py b/aiohttp/client_reqrep.py index 4aeae08ac19..c77213e3ed2 100644 --- a/aiohttp/client_reqrep.py +++ b/aiohttp/client_reqrep.py @@ -244,6 +244,7 @@ class ConnectionKey(NamedTuple): proxy: URL | None proxy_auth: BasicAuth | None proxy_headers_hash: int | None # hash(CIMultiDict) + server_hostname: str | None = None def _is_expected_content_type( @@ -982,6 +983,7 @@ def connection_key(self) -> ConnectionKey: self.proxy, self.proxy_auth, h, + self.server_hostname, ), ) diff --git a/tests/test_client_functional.py b/tests/test_client_functional.py index aa3a6925c2f..6bd57bee612 100644 --- a/tests/test_client_functional.py +++ b/tests/test_client_functional.py @@ -722,6 +722,37 @@ async def handler(request): assert txt == "Test message" +async def test_server_hostname_override_not_reused( + aiohttp_server: AiohttpServer, +) -> None: + """A pooled TLS connection must not be reused for a different server_hostname.""" + trustme = pytest.importorskip("trustme") + + ca = trustme.CA() + cert = ca.issue_cert("first.example") + server_ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + cert.configure_cert(server_ctx) + client_ctx = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH) + ca.configure_trust(client_ctx) + + async def handler(request: web.Request) -> web.Response: + return web.Response(text="ok") + + app = web.Application() + app.router.add_route("GET", "/", handler) + server = await aiohttp_server(app, ssl=server_ctx) + url = server.make_url("/") + + connector = aiohttp.TCPConnector(ssl=client_ctx, limit=1, limit_per_host=1) + async with aiohttp.ClientSession(connector=connector) as session: + async with session.get(url, server_hostname="first.example") as resp: + assert resp.status == 200 + await resp.read() + + with pytest.raises(aiohttp.ClientConnectorCertificateError): + await session.get(url, server_hostname="second.example") + + @pytest.mark.skipif( sys.version_info < (3, 11), reason="ssl_shutdown_timeout requires Python 3.11+" ) diff --git a/tests/test_client_request.py b/tests/test_client_request.py index 66240258620..90ffcecb5d8 100644 --- a/tests/test_client_request.py +++ b/tests/test_client_request.py @@ -1614,6 +1614,22 @@ async def test_connection_key_without_proxy() -> None: await req.close() +async def test_connection_key_includes_server_hostname( + make_request: _RequestMaker, +) -> None: + """A server_hostname override must be part of the connection reuse key.""" + url = URL("https://127.0.0.1:8443/") + none_req = make_request("GET", url) + first = make_request("GET", url, server_hostname="first.example") + first_again = make_request("GET", url, server_hostname="first.example") + second = make_request("GET", url, server_hostname="second.example") + + assert first.connection_key.server_hostname == "first.example" + assert first.connection_key != none_req.connection_key + assert first.connection_key != second.connection_key + assert first.connection_key == first_again.connection_key + + def test_request_info_back_compat() -> None: """Test RequestInfo can be created without real_url.""" url = URL("http://example.com") From 38d16060037e1bfcd6d677abababa3c2a4bb58fa Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 00:30:39 -0500 Subject: [PATCH 28/49] [PR #12825/cb1d6a53 backport][3.14] Scope DigestAuthMiddleware credentials to the request origin (#12839) Co-authored-by: J. Nick Koston Co-authored-by: J. Nick Koston --- CHANGES/12825.bugfix.rst | 1 + aiohttp/client_middleware_digest_auth.py | 21 +++ docs/client_reference.rst | 14 ++ tests/test_client_middleware_digest_auth.py | 170 ++++++++++++++++++++ 4 files changed, 206 insertions(+) create mode 100644 CHANGES/12825.bugfix.rst diff --git a/CHANGES/12825.bugfix.rst b/CHANGES/12825.bugfix.rst new file mode 100644 index 00000000000..88d1bfe8c4c --- /dev/null +++ b/CHANGES/12825.bugfix.rst @@ -0,0 +1 @@ +Scoped :class:`~aiohttp.DigestAuthMiddleware` credentials to the origin of the first request it handles, so a redirect to a different origin no longer triggers a digest response computed from the configured credentials; a challenge from another origin is only answered when that origin falls within a protection space advertised by the anchor origin through the RFC 7616 ``domain`` directive -- by :user:`bdraco`. diff --git a/aiohttp/client_middleware_digest_auth.py b/aiohttp/client_middleware_digest_auth.py index 8151dea5154..6c3e37f7c00 100644 --- a/aiohttp/client_middleware_digest_auth.py +++ b/aiohttp/client_middleware_digest_auth.py @@ -162,6 +162,15 @@ class DigestAuthMiddleware: - Includes replay attack protection with client nonce count tracking - Supports preemptive authentication per RFC 7616 Section 3.6 + Origin scoping: + The credentials are scoped to the origin of the first request the + middleware handles. A request to a different origin is passed through + untouched, so it never receives a digest response computed from those + credentials, unless that origin falls within a protection space the + anchor origin advertised through the RFC 7616 ``domain`` directive. Make + the first request through the middleware against the intended origin, as + the anchor is pinned to it and not reset for the life of the instance. + Standards compliance: - RFC 7616: HTTP Digest Access Authentication (primary reference) - RFC 2617: HTTP Authentication (deprecated by RFC 7616) @@ -198,6 +207,8 @@ def __init__( self._preemptive: bool = preemptive # Set of URLs defining the protection space self._protection_space: list[str] = [] + # Origin the credentials are scoped to; set on the first request. + self._origin: URL | None = None async def _encode(self, method: str, url: URL, body: Payload | Literal[b""]) -> str: """ @@ -447,6 +458,16 @@ async def __call__( self, request: ClientRequest, handler: ClientHandlerType ) -> ClientResponse: """Run the digest auth middleware.""" + # Credentials are scoped to the first request's origin. Other origins + # pass through untouched unless a challenge from the anchor origin + # advertised them via RFC 7616 domain; mirrors aiohttp stripping + # Authorization on cross-origin redirects. + origin = request.url.origin() + if self._origin is None: + self._origin = origin + elif origin != self._origin and not self._in_protection_space(request.url): + return await handler(request) + response = None for retry_count in range(2): # Apply authorization header if: diff --git a/docs/client_reference.rst b/docs/client_reference.rst index 860e6fba119..734201128d6 100644 --- a/docs/client_reference.rst +++ b/docs/client_reference.rst @@ -2447,6 +2447,16 @@ Utilities The server may still respond with a 401 status and ``stale=true`` if the nonce has expired, in which case the middleware will automatically retry with the new nonce. + **Origin scoping** + + The credentials are scoped to the origin of the first request the middleware + handles. A request to a different origin is passed through untouched, so it + never receives a digest response computed from those credentials, unless that + origin falls within a protection space the anchor origin advertised through + the RFC 7616 ``domain`` directive. Make the first request through the + middleware against the intended origin, as the anchor is pinned to it and not + reset for the life of the instance. + To disable preemptive authentication and require a 401 challenge for every request, set ``preemptive=False``:: @@ -2472,6 +2482,10 @@ Utilities .. versionadded:: 3.12 .. versionchanged:: 3.12.8 Added ``preemptive`` parameter to enable/disable preemptive authentication. + .. versionchanged:: 3.14.1 + Credentials are scoped to the origin of the first request the middleware + handles; other origins are passed through untouched unless covered by an + RFC 7616 ``domain`` directive from the anchor origin. .. class:: CookieJar(*, unsafe=False, quote_cookie=True, treat_as_secure_origin = []) diff --git a/tests/test_client_middleware_digest_auth.py b/tests/test_client_middleware_digest_auth.py index 0d2d6ad3325..cdbedee980f 100644 --- a/tests/test_client_middleware_digest_auth.py +++ b/tests/test_client_middleware_digest_auth.py @@ -1198,6 +1198,176 @@ async def handler(request: Request) -> Response: ) # Second request - preemptive auth (entire origin) +async def test_does_not_answer_cross_origin_redirect_challenge( + aiohttp_server: AiohttpServer, +) -> None: + """A cross-origin redirect target must not receive a digest response. + + aiohttp strips the Authorization header on cross-origin redirects; the + digest middleware must not re-add one for the redirect target, otherwise + the configured credentials leak to an origin the caller never targeted. + """ + target_auth_headers: list[str | None] = [] + + async def target_handler(request: Request) -> Response: + auth_header = request.headers.get(hdrs.AUTHORIZATION) + target_auth_headers.append(auth_header) + assert auth_header is None + return Response( + status=401, + headers={ + hdrs.WWW_AUTHENTICATE: 'Digest realm="evil", nonce="cross-origin"' + }, + ) + + target_app = Application() + target_app.router.add_get("/", target_handler) + target_server = await aiohttp_server(target_app) + + async def source_handler(request: Request) -> Response: + return Response( + status=302, headers={hdrs.LOCATION: str(target_server.make_url("/"))} + ) + + source_app = Application() + source_app.router.add_get("/", source_handler) + source_server = await aiohttp_server(source_app) + + digest_auth = DigestAuthMiddleware("victim", "secret") + async with ( + ClientSession(middlewares=(digest_auth,)) as session, + session.get(source_server.make_url("/")) as response, + ): + await response.text() + + assert target_auth_headers == [None] + + +async def test_answers_same_origin_redirect_challenge( + aiohttp_server: AiohttpServer, +) -> None: + """A same-origin redirect that issues a challenge must still authenticate.""" + auth_headers: list[str | None] = [] + + async def handler(request: Request) -> Response: + if request.path == "/start": + return Response(status=302, headers={hdrs.LOCATION: "/protected"}) + auth_header = request.headers.get(hdrs.AUTHORIZATION) + auth_headers.append(auth_header) + if auth_header is None: + return Response( + status=401, + headers={hdrs.WWW_AUTHENTICATE: 'Digest realm="good", nonce="abc"'}, + ) + return Response(text="OK") + + app = Application() + app.router.add_get("/start", handler) + app.router.add_get("/protected", handler) + server = await aiohttp_server(app) + + digest_auth = DigestAuthMiddleware("user", "pass") + async with ( + ClientSession(middlewares=(digest_auth,)) as session, + session.get(server.make_url("/start")) as response, + ): + assert response.status == 200 + assert await response.text() == "OK" + + assert auth_headers[0] is None + assert auth_headers[1] is not None + assert auth_headers[1].startswith("Digest") + + +async def test_answers_cross_origin_within_domain_protection_space( + aiohttp_server: AiohttpServer, +) -> None: + """A different origin advertised via the ``domain`` directive is honored. + + RFC 7616 allows a challenge to define a protection space spanning other + servers through the ``domain`` directive. The anchor origin vouches for + those URIs, so preemptive auth to them is expected. + """ + other_auth_headers: list[str | None] = [] + + async def other_handler(request: Request) -> Response: + other_auth_headers.append(request.headers.get(hdrs.AUTHORIZATION)) + return Response(text="other") + + other_app = Application() + other_app.router.add_get("/", other_handler) + other_server = await aiohttp_server(other_app) + other_origin = str(other_server.make_url("/").origin()) + + async def anchor_handler(request: Request) -> Response: + if request.headers.get(hdrs.AUTHORIZATION) is None: + challenge = f'Digest realm="anchor", nonce="n1", domain="{other_origin}/"' + return Response(status=401, headers={hdrs.WWW_AUTHENTICATE: challenge}) + return Response(text="anchor") + + anchor_app = Application() + anchor_app.router.add_get("/", anchor_handler) + anchor_server = await aiohttp_server(anchor_app) + + digest_auth = DigestAuthMiddleware("user", "pass") + async with ClientSession(middlewares=(digest_auth,)) as session: + async with session.get(anchor_server.make_url("/")) as response: + assert response.status == 200 + async with session.get(other_server.make_url("/")) as response: + assert response.status == 200 + + assert other_auth_headers[0] is not None + assert other_auth_headers[0].startswith("Digest") + + +async def test_does_not_answer_cross_origin_challenge_without_redirect( + aiohttp_server: AiohttpServer, +) -> None: + """Origin scoping applies to any cross-origin request, not just redirects. + + After authenticating against the anchor origin, a direct request to a + different origin that issues its own challenge must not be answered with a + digest response computed from the configured credentials. + """ + other_auth_headers: list[str | None] = [] + + async def other_handler(request: Request) -> Response: + auth_header = request.headers.get(hdrs.AUTHORIZATION) + other_auth_headers.append(auth_header) + assert auth_header is None + return Response( + status=401, + headers={hdrs.WWW_AUTHENTICATE: 'Digest realm="evil", nonce="x"'}, + ) + + other_app = Application() + other_app.router.add_get("/", other_handler) + other_server = await aiohttp_server(other_app) + + async def anchor_handler(request: Request) -> Response: + if request.headers.get(hdrs.AUTHORIZATION) is None: + return Response( + status=401, + headers={hdrs.WWW_AUTHENTICATE: 'Digest realm="anchor", nonce="n1"'}, + ) + return Response(text="anchor") + + anchor_app = Application() + anchor_app.router.add_get("/", anchor_handler) + anchor_server = await aiohttp_server(anchor_app) + + digest_auth = DigestAuthMiddleware("user", "pass") + async with ClientSession(middlewares=(digest_auth,)) as session: + async with session.get(anchor_server.make_url("/")) as response: + assert response.status == 200 + async with session.get(other_server.make_url("/")) as response: + assert response.status == 401 + + # The other origin only ever saw the unauthenticated request; the + # middleware never answered its challenge. + assert other_auth_headers == [None] + + @pytest.mark.parametrize( ("status", "headers", "expected"), [ From 90ea5526c1008d9c0f4779b506be35ebf3251095 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 00:30:53 -0500 Subject: [PATCH 29/49] [PR #12831/1ac92dae backport][3.15] Payload close on disconnect (#12844) Co-authored-by: J. Nick Koston --- CHANGES/12831.bugfix.rst | 1 + aiohttp/web_response.py | 6 ++-- tests/test_web_response.py | 73 +++++++++++++++++++++++++++++++++++++- 3 files changed, 77 insertions(+), 3 deletions(-) create mode 100644 CHANGES/12831.bugfix.rst diff --git a/CHANGES/12831.bugfix.rst b/CHANGES/12831.bugfix.rst new file mode 100644 index 00000000000..bf460ffccac --- /dev/null +++ b/CHANGES/12831.bugfix.rst @@ -0,0 +1 @@ +Fixed :meth:`aiohttp.web.Response.write_eof` skipping ``Payload.close()`` when the body write was interrupted by an error or cancellation, for example when a client disconnects mid-response; the payload close hook now runs in a ``finally`` so a :class:`~aiohttp.payload.Payload` body always releases its resources -- by :user:`bdraco`. diff --git a/aiohttp/web_response.py b/aiohttp/web_response.py index daf29eccce4..cbe4985cfb9 100644 --- a/aiohttp/web_response.py +++ b/aiohttp/web_response.py @@ -808,8 +808,10 @@ async def write_eof(self, data: bytes = b"") -> None: if body is None or self._must_be_empty_body: await super().write_eof() elif isinstance(self._body, Payload): - await self._body.write(self._payload_writer) - await self._body.close() + try: + await self._body.write(self._payload_writer) + finally: + await self._body.close() await super().write_eof() else: await super().write_eof(cast(bytes, body)) diff --git a/tests/test_web_response.py b/tests/test_web_response.py index 52a574f1c0c..c5ca849b6b9 100644 --- a/tests/test_web_response.py +++ b/tests/test_web_response.py @@ -1,3 +1,4 @@ +import asyncio import collections.abc import datetime import gzip @@ -17,7 +18,7 @@ from aiohttp.helpers import ETag from aiohttp.http_writer import StreamWriter, _serialize_headers from aiohttp.multipart import BodyPartReader, MultipartWriter -from aiohttp.payload import BytesPayload, StringPayload +from aiohttp.payload import BytesPayload, Payload, StringPayload from aiohttp.test_utils import make_mocked_request from aiohttp.web import ( ContentCoding, @@ -1434,6 +1435,76 @@ async def test_consecutive_write_eof() -> None: writer.write_eof.assert_called_once_with(data) +class _ClosingPayload(Payload): + """Payload test double that records whether close() ran.""" + + def __init__(self) -> None: + super().__init__(None) + self.close_called = False + self.started = asyncio.Event() + self.release = asyncio.Event() + self.fail = False + + async def write(self, writer: AbstractStreamWriter) -> None: + self.started.set() + if self.fail: + raise ConnectionResetError("client gone") + await self.release.wait() + + async def close(self) -> None: + self.close_called = True + await super().close() + + def decode(self, encoding: str = "utf-8", errors: str = "strict") -> str: + assert False + + +async def test_write_eof_closes_payload_on_success() -> None: + writer = mock.create_autospec(AbstractStreamWriter, spec_set=True, instance=True) + req = make_request("GET", "/", writer=writer) + payload = _ClosingPayload() + payload.release.set() + resp = web.Response(body=payload) + + await resp.prepare(req) + await resp.write_eof() + + assert payload.close_called + assert writer.write_eof.called + + +async def test_write_eof_closes_payload_on_write_error() -> None: + writer = mock.create_autospec(AbstractStreamWriter, spec_set=True, instance=True) + req = make_request("GET", "/", writer=writer) + payload = _ClosingPayload() + payload.fail = True + resp = web.Response(body=payload) + + await resp.prepare(req) + with pytest.raises(ConnectionResetError): + await resp.write_eof() + + assert payload.close_called + assert not writer.write_eof.called + + +async def test_write_eof_closes_payload_on_cancel() -> None: + writer = mock.create_autospec(AbstractStreamWriter, spec_set=True, instance=True) + req = make_request("GET", "/", writer=writer) + payload = _ClosingPayload() + resp = web.Response(body=payload) + + await resp.prepare(req) + task = asyncio.ensure_future(resp.write_eof()) + await payload.started.wait() + task.cancel() + with pytest.raises(asyncio.CancelledError): + await task + + assert payload.close_called + assert not writer.write_eof.called + + def test_set_text_with_content_type() -> None: resp = Response() resp.content_type = "text/html" From 2475b63d7764039232622659a3b94c55977e6f34 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:31:25 -0500 Subject: [PATCH 30/49] [PR #12828/13b635d7 backport][3.15] Bounded unread compressed drain (#12846) --- CHANGES/12828.bugfix.rst | 1 + aiohttp/streams.py | 17 +++++++--- tests/test_streams.py | 28 ++++++++++++++++ tests/test_web_functional.py | 63 ++++++++++++++++++++++++++++++++++++ 4 files changed, 104 insertions(+), 5 deletions(-) create mode 100644 CHANGES/12828.bugfix.rst diff --git a/CHANGES/12828.bugfix.rst b/CHANGES/12828.bugfix.rst new file mode 100644 index 00000000000..9893577a587 --- /dev/null +++ b/CHANGES/12828.bugfix.rst @@ -0,0 +1 @@ +Fixed :meth:`~aiohttp.StreamReader.readany` and :meth:`~aiohttp.StreamReader.read_nowait` joining data fed back into the buffer during the call (when draining below the low water mark resumes reading) into a single unbounded :class:`bytes`; a call now returns only the chunks that were buffered when it started, keeping the drain of an unread auto-decompressed request body bounded by the read buffer -- by :user:`bdraco`. diff --git a/aiohttp/streams.py b/aiohttp/streams.py index 196469c005a..e1a5b531470 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -560,14 +560,21 @@ def _read_nowait(self, n: int) -> bytes: """Read not more than n bytes, or whole buffer if n == -1""" self._timer.assert_timeout() - chunks = [] + if n == -1: + # Drain only chunks present now; _read_nowait_chunk() can + # re-entrantly resume_reading() and refill the buffer. + count = len(self._buffer) + if count == 1: + return self._read_nowait_chunk(-1) + return b"".join([self._read_nowait_chunk(-1) for _ in range(count)]) + + chunks: list[bytes] = [] while self._buffer: chunk = self._read_nowait_chunk(n) chunks.append(chunk) - if n != -1: - n -= len(chunk) - if n == 0: - break + n -= len(chunk) + if n == 0: + break return b"".join(chunks) if chunks else b"" diff --git a/tests/test_streams.py b/tests/test_streams.py index ae3c731e477..8c43951ab86 100644 --- a/tests/test_streams.py +++ b/tests/test_streams.py @@ -1723,3 +1723,31 @@ def resume_reading() -> None: protocol.resume_reading.assert_called() assert protocol._reading_paused is False + + +async def test_readany_does_not_drain_reentrant_refill( + protocol: mock.Mock, +) -> None: + """A single readany() must not reassemble data fed re-entrantly. + + Draining below the low water mark resumes reading, which can synchronously + refill the buffer (e.g. decompressing another chunk). Joining that refill in + one call would reassemble an unbounded body. + """ + loop = asyncio.get_running_loop() + stream = streams.StreamReader(protocol, limit=4, loop=loop) + + refills = [b"second", b"third"] + + def resume_reading() -> None: + if refills: + stream.feed_data(refills.pop(0)) + + protocol.resume_reading.side_effect = resume_reading + + stream.feed_data(b"first") + + # Popping "first" refills "second", but this readany() returns only "first". + assert await stream.readany() == b"first" + assert await stream.readany() == b"second" + assert await stream.readany() == b"third" diff --git a/tests/test_web_functional.py b/tests/test_web_functional.py index f054f71db21..231a79c2757 100644 --- a/tests/test_web_functional.py +++ b/tests/test_web_functional.py @@ -4,7 +4,9 @@ import pathlib import socket import sys +import zlib from collections.abc import Generator +from contextlib import suppress from typing import Any, NoReturn from unittest import mock @@ -24,7 +26,9 @@ ) from aiohttp.compression_utils import ZLibBackend, ZLibCompressObjProtocol from aiohttp.hdrs import CONTENT_LENGTH, CONTENT_TYPE, TRANSFER_ENCODING +from aiohttp.helpers import DEFAULT_CHUNK_SIZE from aiohttp.pytest_plugin import AiohttpClient, AiohttpServer +from aiohttp.streams import StreamReader from aiohttp.typedefs import Handler from aiohttp.web_protocol import RequestHandler @@ -1711,6 +1715,65 @@ async def handler(request): await resp.release() +@pytest.mark.parametrize("decompressed_size", [4 * 1024 * 1024, 32 * 1024 * 1024]) +async def test_unread_compressed_body_drain_is_bounded( + aiohttp_server: AiohttpServer, + monkeypatch: pytest.MonkeyPatch, + decompressed_size: int, +) -> None: + """Draining an unread compressed body stays bounded by the read buffer. + + A handler that rejects before reading still drains the payload during + lingering close; a small compressed body must not force a large transient + allocation (a deflate-bomb style DoS). + """ + drain_reads: list[int] = [] + drained = asyncio.Event() + readany = StreamReader.readany + + async def record_readany(self: StreamReader) -> bytes: + data = await readany(self) + assert data + drain_reads.append(len(data)) + drained.set() + return data + + monkeypatch.setattr(StreamReader, "readany", record_readany) + + async def handler(request: web.Request) -> web.Response: + return web.Response(status=401) + + app = web.Application(client_max_size=1024) + app.router.add_post("/", handler) + server = await aiohttp_server(app) + + body = zlib.compress(b"a" * decompressed_size) + assert len(body) < decompressed_size + head = ( + b"POST / HTTP/1.1\r\n" + b"Host: localhost\r\n" + b"Content-Encoding: deflate\r\n" + b"Content-Length: %d\r\n" + b"Connection: keep-alive\r\n\r\n" + ) % len(body) + + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write(head + body) + await writer.drain() + status_line = await asyncio.wait_for(reader.readline(), 5) + assert status_line.startswith(b"HTTP/1.1 401 ") + await asyncio.wait_for(drained.wait(), 5) + finally: + writer.close() + with suppress(ConnectionResetError, BrokenPipeError): + await writer.wait_closed() + + # Bounded by the buffer, not the decompressed size. + assert max(drain_reads) <= 3 * DEFAULT_CHUNK_SIZE + assert max(drain_reads) < decompressed_size + + async def test_app_max_client_size(aiohttp_client) -> None: async def handler(request): await request.post() From 3912667ae5281b5a14c58e04e02a3834534ff0d5 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:31:45 -0500 Subject: [PATCH 31/49] [3.14] Add test that env proxy auth is scoped to the redirect-selected proxy (#12842) --- tests/test_proxy_functional.py | 56 +++++++++++++++++++++++++++++++++- 1 file changed, 55 insertions(+), 1 deletion(-) diff --git a/tests/test_proxy_functional.py b/tests/test_proxy_functional.py index 85309dc5cea..33cd31872c9 100644 --- a/tests/test_proxy_functional.py +++ b/tests/test_proxy_functional.py @@ -12,13 +12,14 @@ from uuid import uuid4 import pytest +from multidict import CIMultiDict from yarl import URL import aiohttp from aiohttp import ClientResponse, web from aiohttp.client_exceptions import ClientConnectionError from aiohttp.helpers import IS_MACOS, IS_WINDOWS -from aiohttp.pytest_plugin import AiohttpServer +from aiohttp.pytest_plugin import AiohttpRawServer, AiohttpServer from aiohttp.test_utils import TestServer if TYPE_CHECKING: @@ -849,6 +850,59 @@ async def test_proxy_from_env_http_without_auth_from_wrong_netrc( assert "Proxy-Authorization" not in proxy.request.headers +async def test_proxy_from_env_auth_scoped_to_redirect_selected_proxy( + aiohttp_raw_server: AiohttpRawServer, + get_request, + monkeypatch: pytest.MonkeyPatch, + tmp_path: pathlib.Path, +) -> None: + # Redirect from an authenticated http_proxy to an HTTPS target served by a + # separate https_proxy must not leak the first proxy's Proxy-Authorization + # onto the second proxy's CONNECT request. + auth_header = aiohttp.encode_basic_auth("user", "pass") + http_proxy_requests: list[CIMultiDict[str]] = [] + https_proxy_requests: list[CIMultiDict[str]] = [] + + async def http_proxy_handler(request: web.Request) -> web.Response: + http_proxy_requests.append(CIMultiDict(request.headers)) + return web.Response( + status=302, headers={"Location": "https://attacker.example/secret"} + ) + + async def https_proxy_handler(request: web.Request) -> web.Response: + https_proxy_requests.append(CIMultiDict(request.headers)) + return web.Response(status=502) + + http_proxy = await aiohttp_raw_server(http_proxy_handler) + https_proxy = await aiohttp_raw_server(https_proxy_handler) + + for name in ( + "http_proxy", + "https_proxy", + "all_proxy", + "no_proxy", + "HTTP_PROXY", + "HTTPS_PROXY", + "ALL_PROXY", + "NO_PROXY", + ): + monkeypatch.delenv(name, raising=False) + netrc_file = tmp_path / "empty_netrc" + netrc_file.write_text("") + http_proxy_url = http_proxy.make_url("/").with_user("user").with_password("pass") + monkeypatch.setenv("http_proxy", str(http_proxy_url)) + monkeypatch.setenv("https_proxy", str(https_proxy.make_url("/"))) + monkeypatch.setenv("NETRC", str(netrc_file)) + + with pytest.raises(aiohttp.ClientHttpProxyError): + await get_request(url="http://victim.example/redirect", trust_env=True) + + assert len(http_proxy_requests) == 1 + assert http_proxy_requests[0]["Proxy-Authorization"] == auth_header + assert len(https_proxy_requests) == 1 + assert "Proxy-Authorization" not in https_proxy_requests[0] + + @pytest.mark.xfail async def xtest_proxy_from_env_https(proxy_test_server, get_request, mocker): url = "https://aiohttp.io/path" From 58fc08e4d72f768c7bb25b7c83933f5366524c7e Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:31:55 -0500 Subject: [PATCH 32/49] [3.15] Add test that env proxy auth is scoped to the redirect-selected proxy (#12841) --- tests/test_proxy_functional.py | 56 +++++++++++++++++++++++++++++++++- 1 file changed, 55 insertions(+), 1 deletion(-) diff --git a/tests/test_proxy_functional.py b/tests/test_proxy_functional.py index 85309dc5cea..33cd31872c9 100644 --- a/tests/test_proxy_functional.py +++ b/tests/test_proxy_functional.py @@ -12,13 +12,14 @@ from uuid import uuid4 import pytest +from multidict import CIMultiDict from yarl import URL import aiohttp from aiohttp import ClientResponse, web from aiohttp.client_exceptions import ClientConnectionError from aiohttp.helpers import IS_MACOS, IS_WINDOWS -from aiohttp.pytest_plugin import AiohttpServer +from aiohttp.pytest_plugin import AiohttpRawServer, AiohttpServer from aiohttp.test_utils import TestServer if TYPE_CHECKING: @@ -849,6 +850,59 @@ async def test_proxy_from_env_http_without_auth_from_wrong_netrc( assert "Proxy-Authorization" not in proxy.request.headers +async def test_proxy_from_env_auth_scoped_to_redirect_selected_proxy( + aiohttp_raw_server: AiohttpRawServer, + get_request, + monkeypatch: pytest.MonkeyPatch, + tmp_path: pathlib.Path, +) -> None: + # Redirect from an authenticated http_proxy to an HTTPS target served by a + # separate https_proxy must not leak the first proxy's Proxy-Authorization + # onto the second proxy's CONNECT request. + auth_header = aiohttp.encode_basic_auth("user", "pass") + http_proxy_requests: list[CIMultiDict[str]] = [] + https_proxy_requests: list[CIMultiDict[str]] = [] + + async def http_proxy_handler(request: web.Request) -> web.Response: + http_proxy_requests.append(CIMultiDict(request.headers)) + return web.Response( + status=302, headers={"Location": "https://attacker.example/secret"} + ) + + async def https_proxy_handler(request: web.Request) -> web.Response: + https_proxy_requests.append(CIMultiDict(request.headers)) + return web.Response(status=502) + + http_proxy = await aiohttp_raw_server(http_proxy_handler) + https_proxy = await aiohttp_raw_server(https_proxy_handler) + + for name in ( + "http_proxy", + "https_proxy", + "all_proxy", + "no_proxy", + "HTTP_PROXY", + "HTTPS_PROXY", + "ALL_PROXY", + "NO_PROXY", + ): + monkeypatch.delenv(name, raising=False) + netrc_file = tmp_path / "empty_netrc" + netrc_file.write_text("") + http_proxy_url = http_proxy.make_url("/").with_user("user").with_password("pass") + monkeypatch.setenv("http_proxy", str(http_proxy_url)) + monkeypatch.setenv("https_proxy", str(https_proxy.make_url("/"))) + monkeypatch.setenv("NETRC", str(netrc_file)) + + with pytest.raises(aiohttp.ClientHttpProxyError): + await get_request(url="http://victim.example/redirect", trust_env=True) + + assert len(http_proxy_requests) == 1 + assert http_proxy_requests[0]["Proxy-Authorization"] == auth_header + assert len(https_proxy_requests) == 1 + assert "Proxy-Authorization" not in https_proxy_requests[0] + + @pytest.mark.xfail async def xtest_proxy_from_env_https(proxy_test_server, get_request, mocker): url = "https://aiohttp.io/path" From b40867f82e66ecaffc90b3c447904e0674f265e8 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:32:46 -0500 Subject: [PATCH 33/49] [PR #12826/36df6c13 backport][3.15] Enforce max_line_size on fragmented request target and reason in C parser (#12838) --- CHANGES/12826.bugfix.rst | 1 + aiohttp/_http_parser.pyx | 4 ++-- tests/test_http_parser.py | 22 ++++++++++++++++++++++ 3 files changed, 25 insertions(+), 2 deletions(-) create mode 100644 CHANGES/12826.bugfix.rst diff --git a/CHANGES/12826.bugfix.rst b/CHANGES/12826.bugfix.rst new file mode 100644 index 00000000000..7e095615d84 --- /dev/null +++ b/CHANGES/12826.bugfix.rst @@ -0,0 +1 @@ +Fixed the C HTTP parser not enforcing ``max_line_size`` on a request target or response reason phrase that is split across multiple reads; each fragment was checked on its own, so an accumulated line could exceed the limit without raising ``LineTooLong``. The accumulated length is now checked, matching the pure-Python parser -- by :user:`bdraco`. diff --git a/aiohttp/_http_parser.pyx b/aiohttp/_http_parser.pyx index 8bf16ab16f4..b5e5d6fc375 100644 --- a/aiohttp/_http_parser.pyx +++ b/aiohttp/_http_parser.pyx @@ -779,7 +779,7 @@ cdef int cb_on_url(cparser.llhttp_t* parser, const char *at, size_t length) except -1: cdef HttpParser pyparser = parser.data try: - if length > pyparser._max_line_size: + if len(pyparser._buf) + length > pyparser._max_line_size: status = pyparser._buf + at[:length] raise LineTooLong(status[:100] + b"...", pyparser._max_line_size) extend(pyparser._buf, at, length) @@ -794,7 +794,7 @@ cdef int cb_on_status(cparser.llhttp_t* parser, const char *at, size_t length) except -1: cdef HttpParser pyparser = parser.data try: - if length > pyparser._max_line_size: + if len(pyparser._buf) + length > pyparser._max_line_size: reason = pyparser._buf + at[:length] raise LineTooLong(reason[:100] + b"...", pyparser._max_line_size) extend(pyparser._buf, at, length) diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index b9125fa4078..9b7fa83d9a6 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -1574,6 +1574,17 @@ def test_http_request_max_status_line_under_limit(parser: HttpRequestParser) -> assert msg.url == URL("/path" + path.decode()) +def test_http_request_max_status_line_fragmented( + parser: HttpRequestParser, +) -> None: + # Split an overlong request target across reads so that each callback + # fragment is under the limit but the accumulated target is not. + match = "400, message:\n Got more than 8190 bytes when reading" + with pytest.raises(http_exceptions.LineTooLong, match=match): + parser.feed_data(b"GET /" + b"a" * 8000) + parser.feed_data(b"a" * 8000 + b" HTTP/1.1\r\nHost: a\r\n\r\n") + + def test_http_response_parser_utf8(response) -> None: text = "HTTP/1.1 200 Ok\r\nx-test:тест\r\n\r\n".encode() @@ -1651,6 +1662,17 @@ def test_http_response_parser_status_line_under_limit( assert msg.reason == reason.decode() +def test_http_response_parser_status_line_too_long_fragmented( + response: HttpResponseParser, +) -> None: + # Split an overlong reason phrase across reads so that each callback + # fragment is under the limit but the accumulated reason is not. + match = "400, message:\n Got more than 8190 bytes when reading" + with pytest.raises(http_exceptions.LineTooLong, match=match): + response.feed_data(b"HTTP/1.1 200 " + b"a" * 8000) + response.feed_data(b"a" * 8000 + b"\r\n\r\n") + + def test_http_response_parser_bad_version(response) -> None: with pytest.raises(http_exceptions.BadHttpMessage): response.feed_data(b"HT/11 200 Ok\r\n\r\n") From 5ab61bb4cd88f19b712f12c7c9295fe262bf804d Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:33:03 -0500 Subject: [PATCH 34/49] [PR #12826/36df6c13 backport][3.14] Enforce max_line_size on fragmented request target and reason in C parser (#12837) --- CHANGES/12826.bugfix.rst | 1 + aiohttp/_http_parser.pyx | 4 ++-- tests/test_http_parser.py | 22 ++++++++++++++++++++++ 3 files changed, 25 insertions(+), 2 deletions(-) create mode 100644 CHANGES/12826.bugfix.rst diff --git a/CHANGES/12826.bugfix.rst b/CHANGES/12826.bugfix.rst new file mode 100644 index 00000000000..7e095615d84 --- /dev/null +++ b/CHANGES/12826.bugfix.rst @@ -0,0 +1 @@ +Fixed the C HTTP parser not enforcing ``max_line_size`` on a request target or response reason phrase that is split across multiple reads; each fragment was checked on its own, so an accumulated line could exceed the limit without raising ``LineTooLong``. The accumulated length is now checked, matching the pure-Python parser -- by :user:`bdraco`. diff --git a/aiohttp/_http_parser.pyx b/aiohttp/_http_parser.pyx index 8bf16ab16f4..b5e5d6fc375 100644 --- a/aiohttp/_http_parser.pyx +++ b/aiohttp/_http_parser.pyx @@ -779,7 +779,7 @@ cdef int cb_on_url(cparser.llhttp_t* parser, const char *at, size_t length) except -1: cdef HttpParser pyparser = parser.data try: - if length > pyparser._max_line_size: + if len(pyparser._buf) + length > pyparser._max_line_size: status = pyparser._buf + at[:length] raise LineTooLong(status[:100] + b"...", pyparser._max_line_size) extend(pyparser._buf, at, length) @@ -794,7 +794,7 @@ cdef int cb_on_status(cparser.llhttp_t* parser, const char *at, size_t length) except -1: cdef HttpParser pyparser = parser.data try: - if length > pyparser._max_line_size: + if len(pyparser._buf) + length > pyparser._max_line_size: reason = pyparser._buf + at[:length] raise LineTooLong(reason[:100] + b"...", pyparser._max_line_size) extend(pyparser._buf, at, length) diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index b9125fa4078..9b7fa83d9a6 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -1574,6 +1574,17 @@ def test_http_request_max_status_line_under_limit(parser: HttpRequestParser) -> assert msg.url == URL("/path" + path.decode()) +def test_http_request_max_status_line_fragmented( + parser: HttpRequestParser, +) -> None: + # Split an overlong request target across reads so that each callback + # fragment is under the limit but the accumulated target is not. + match = "400, message:\n Got more than 8190 bytes when reading" + with pytest.raises(http_exceptions.LineTooLong, match=match): + parser.feed_data(b"GET /" + b"a" * 8000) + parser.feed_data(b"a" * 8000 + b" HTTP/1.1\r\nHost: a\r\n\r\n") + + def test_http_response_parser_utf8(response) -> None: text = "HTTP/1.1 200 Ok\r\nx-test:тест\r\n\r\n".encode() @@ -1651,6 +1662,17 @@ def test_http_response_parser_status_line_under_limit( assert msg.reason == reason.decode() +def test_http_response_parser_status_line_too_long_fragmented( + response: HttpResponseParser, +) -> None: + # Split an overlong reason phrase across reads so that each callback + # fragment is under the limit but the accumulated reason is not. + match = "400, message:\n Got more than 8190 bytes when reading" + with pytest.raises(http_exceptions.LineTooLong, match=match): + response.feed_data(b"HTTP/1.1 200 " + b"a" * 8000) + response.feed_data(b"a" * 8000 + b"\r\n\r\n") + + def test_http_response_parser_bad_version(response) -> None: with pytest.raises(http_exceptions.BadHttpMessage): response.feed_data(b"HT/11 200 Ok\r\n\r\n") From 95eb19f5eb686fe6a809358294256422a1ab148c Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:33:31 -0500 Subject: [PATCH 35/49] [PR #12824/60b85e98 backport][3.15] Preserve host-only cookie scope across CookieJar save/load (#12834) --- CHANGES/12824.bugfix.rst | 1 + aiohttp/cookiejar.py | 52 ++++++++++----- tests/test_cookiejar.py | 135 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 171 insertions(+), 17 deletions(-) create mode 100644 CHANGES/12824.bugfix.rst diff --git a/CHANGES/12824.bugfix.rst b/CHANGES/12824.bugfix.rst new file mode 100644 index 00000000000..f8dbd169c31 --- /dev/null +++ b/CHANGES/12824.bugfix.rst @@ -0,0 +1 @@ +Fixed :class:`~aiohttp.CookieJar` dropping the host-only flag of cookies when persisted with :meth:`~aiohttp.CookieJar.save` and reloaded with :meth:`~aiohttp.CookieJar.load`, so a cookie set without a ``Domain`` attribute is again scoped to the exact host that set it after a reload; the absolute expiration deadline is now persisted as well, so a reloaded cookie keeps its original lifetime instead of being rescheduled from the load time. :meth:`~aiohttp.CookieJar.load` now replaces the jar contents rather than merging onto prior state, and loaded cookies pass through the same acceptance rules as :meth:`~aiohttp.CookieJar.update_cookies`, so a cookie for an IP-address host is dropped when loaded into a jar created without ``unsafe=True`` -- by :user:`bdraco`. diff --git a/aiohttp/cookiejar.py b/aiohttp/cookiejar.py index e1579c0ed4c..2b972247009 100644 --- a/aiohttp/cookiejar.py +++ b/aiohttp/cookiejar.py @@ -39,6 +39,9 @@ _MIN_SCHEDULED_COOKIE_EXPIRATION = 100 _SIMPLE_COOKIE = SimpleCookie() +# Not persisted; the absolute deadline is saved instead. +_RELATIVE_EXPIRY_ATTRS = frozenset(("max-age", "expires")) + class _RestrictedCookieUnpickler(pickle._Unpickler): """A restricted unpickler that only allows cookie-related types. @@ -174,21 +177,28 @@ def save(self, file_path: PathLike) -> None: :class:`str` or :class:`pathlib.Path` instance. """ file_path = pathlib.Path(file_path) - data: dict[str, dict[str, dict[str, str | bool]]] = {} + data: dict[str, dict[str, dict[str, str | bool | float]]] = {} for (domain, path), cookie in self._cookies.items(): key = f"{domain}|{path}" data[key] = {} for name, morsel in cookie.items(): - morsel_data: dict[str, str | bool] = { + morsel_data: dict[str, str | bool | float] = { "key": morsel.key, "value": morsel.value, "coded_value": morsel.coded_value, } - # Save all morsel attributes that have values + # Skip relative expiry; the absolute deadline is saved below. for attr in morsel._reserved: # type: ignore[attr-defined] + if attr in _RELATIVE_EXPIRY_ATTRS: + continue attr_val = morsel[attr] if attr_val: morsel_data[attr] = attr_val + # Persist or it reloads as a domain cookie and leaks to subdomains. + if (domain, name) in self._host_only_cookies: + morsel_data["host_only"] = True + if (exp := self._expirations.get((domain, path, name))) is not None: + morsel_data["expires_timestamp"] = exp data[key][name] = morsel_data # Cookie persistence may include authentication/session tokens. @@ -209,6 +219,9 @@ def load(self, file_path: PathLike) -> None: pickle format (using a restricted unpickler) for backward compatibility with existing cookie files. + Replaces the current jar contents; loaded cookies pass through the + same acceptance rules as :meth:`update_cookies`. + :param file_path: Path to file from where cookies will be imported, :class:`str` or :class:`pathlib.Path` instance. """ @@ -217,32 +230,28 @@ def load(self, file_path: PathLike) -> None: try: with file_path.open(mode="r", encoding="utf-8") as f: data = json.load(f) - self._cookies = self._load_json_data(data) + self._load_json_data(data) except (json.JSONDecodeError, UnicodeDecodeError, ValueError): # Fall back to legacy pickle format with restricted unpickler with file_path.open(mode="rb") as f: self._cookies = _RestrictedCookieUnpickler(f).load() def _load_json_data( - self, data: dict[str, dict[str, dict[str, str | bool]]] - ) -> defaultdict[tuple[str, str], SimpleCookie]: - """Load cookies from parsed JSON data.""" - cookies: defaultdict[tuple[str, str], SimpleCookie] = defaultdict(SimpleCookie) + self, data: dict[str, dict[str, dict[str, str | bool | float]]] + ) -> None: + """Replace contents, routing cookies through update_cookies().""" + self.clear() for compound_key, cookie_data in data.items(): domain, path = compound_key.split("|", 1) - key = (domain, path) for name, morsel_data in cookie_data.items(): morsel: Morsel[str] = Morsel() - morsel_key = morsel_data["key"] - morsel_value = morsel_data["value"] - morsel_coded_value = morsel_data["coded_value"] # Use __setstate__ to bypass validation, same pattern # used in _build_morsel and _cookie_helpers. morsel.__setstate__( # type: ignore[attr-defined] { - "key": morsel_key, - "value": morsel_value, - "coded_value": morsel_coded_value, + "key": morsel_data["key"], + "value": morsel_data["value"], + "coded_value": morsel_data["coded_value"], } ) # Restore morsel attributes @@ -253,8 +262,17 @@ def _load_json_data( "coded_value", ): morsel[attr] = morsel_data[attr] - cookies[key][name] = morsel - return cookies + # Drop the domain so update_cookies() re-marks it host-only. + if morsel_data.get("host_only"): + morsel["domain"] = "" + response_url = ( + URL.build(scheme="https", host=domain) if domain else URL() + ) + self.update_cookies({name: morsel}, response_url) + # Restore the absolute deadline; update_cookies() schedules none. + if (exp := morsel_data.get("expires_timestamp")) is not None: + self._expire_cookie(float(exp), domain, path, name) + self._do_expiration() def clear(self, predicate: ClearCookiePredicate | None = None) -> None: if predicate is None: diff --git a/tests/test_cookiejar.py b/tests/test_cookiejar.py index fa1e31cd9f6..9276ff252dc 100644 --- a/tests/test_cookiejar.py +++ b/tests/test_cookiejar.py @@ -2,6 +2,7 @@ import datetime import heapq import itertools +import json import logging import os import pathlib @@ -1814,6 +1815,140 @@ async def test_save_load_json_partitioned_cookies(tmp_path: Path) -> None: assert s["path"] == lo["path"] +async def test_save_load_json_preserves_host_only_scope(tmp_path: Path) -> None: + """Verify save/load keeps host-only cookies off subdomains.""" + file_path = tmp_path / "host_only.json" + issuer = URL("https://auth.example.com/login") + subdomain = URL("https://sub.auth.example.com/") + + jar_save = CookieJar() + jar_save.update_cookies({"sid": "hostonly"}, response_url=issuer) + assert "sid" not in jar_save.filter_cookies(subdomain) + jar_save.save(file_path=file_path) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + assert jar_load.host_only_cookies == frozenset({("auth.example.com", "sid")}) + assert "sid" not in jar_load.filter_cookies(subdomain) + assert "sid" in jar_load.filter_cookies(issuer) + + +async def test_save_load_json_domain_cookie_still_matches_subdomain( + tmp_path: Path, +) -> None: + """Verify save/load keeps an explicit Domain cookie valid for subdomains.""" + file_path = tmp_path / "domain.json" + subdomain = URL("https://sub.example.com/") + + jar_save = CookieJar() + jar_save.update_cookies_from_headers( + ["sid=domaincookie; Domain=example.com"], URL("https://example.com/") + ) + jar_save.save(file_path=file_path) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + assert jar_load.host_only_cookies == frozenset() + assert "sid" in jar_load.filter_cookies(subdomain) + + +async def test_save_load_json_preserves_max_age_deadline(tmp_path: Path) -> None: + """Verify save/load restores the absolute deadline without resetting it.""" + file_path = tmp_path / "max_age.json" + url = URL("https://example.com/") + + jar_save = CookieJar() + jar_save.update_cookies_from_headers( + ["sid=x; Max-Age=3600; Domain=example.com"], url + ) + expirations = dict(jar_save._expirations) + jar_save.save(file_path=file_path) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + # The deadline is restored as the original absolute time, not now + Max-Age. + assert dict(jar_load._expirations) == expirations + assert "sid" in jar_load.filter_cookies(url) + + +async def test_save_load_json_drops_expired_cookie(tmp_path: Path) -> None: + """Verify a cookie whose persisted deadline is in the past is dropped on load.""" + file_path = tmp_path / "expired.json" + url = URL("https://example.com/") + + # Save a future-expiring cookie, then rewrite its persisted deadline to the + # past so the cookie survives save() and the drop happens on the load path. + jar_save = CookieJar() + jar_save.update_cookies_from_headers( + ["sid=x; Expires=Tue, 1 Jan 2999 12:00:00 GMT; Domain=example.com"], url + ) + jar_save.save(file_path=file_path) + data = json.loads(file_path.read_text()) + _, cookies = next(iter(data.items())) + cookies["sid"]["expires_timestamp"] = 0.0 + file_path.write_text(json.dumps(data)) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + assert len(jar_load) == 0 + assert "sid" not in jar_load.filter_cookies(url) + + +async def test_save_load_json_preserves_expires_deadline(tmp_path: Path) -> None: + """Verify a future Expires deadline survives a save/load roundtrip.""" + file_path = tmp_path / "expires.json" + url = URL("https://example.com/") + + jar_save = CookieJar() + jar_save.update_cookies_from_headers( + ["sid=x; Expires=Tue, 1 Jan 2999 12:00:00 GMT; Domain=example.com"], url + ) + expirations = dict(jar_save._expirations) + jar_save.save(file_path=file_path) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + assert dict(jar_load._expirations) == expirations + assert "sid" in jar_load.filter_cookies(url) + + +async def test_load_json_old_format_without_new_keys(tmp_path: Path) -> None: + """Verify a file written by an older version (no host_only/expires_timestamp) loads.""" + file_path = tmp_path / "old.json" + # Old schema: no host_only, no expires_timestamp; relative max-age morsel attr. + file_path.write_text( + json.dumps( + { + "example.com|/": { + "sid": { + "key": "sid", + "value": "x", + "coded_value": "x", + "domain": "example.com", + "max-age": "3600", + } + } + } + ) + ) + url = URL("https://example.com/") + + jar_load = CookieJar() + # No exception when the new keys are absent. + jar_load.load(file_path=file_path) + + # A host-only cookie saved without Domain by an older version had no domain + # field, so it now loads as a domain cookie (the documented migration loss). + assert "sid" in jar_load.filter_cookies(url) + # max-age is rescheduled from load time rather than an absolute deadline. + assert any(key[2] == "sid" for key in jar_load._expirations) + + async def test_json_format_is_safe(tmp_path: Path) -> None: """Verify the JSON file format cannot execute code on load.""" import json From 4f7480e474cccc6a8cc2c92ad3f17a31dedf8232 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:39:29 -0500 Subject: [PATCH 36/49] [PR #12828/13b635d7 backport][3.14] Bounded unread compressed drain (#12845) --- CHANGES/12828.bugfix.rst | 1 + aiohttp/streams.py | 17 +++++++--- tests/test_streams.py | 28 ++++++++++++++++ tests/test_web_functional.py | 63 ++++++++++++++++++++++++++++++++++++ 4 files changed, 104 insertions(+), 5 deletions(-) create mode 100644 CHANGES/12828.bugfix.rst diff --git a/CHANGES/12828.bugfix.rst b/CHANGES/12828.bugfix.rst new file mode 100644 index 00000000000..9893577a587 --- /dev/null +++ b/CHANGES/12828.bugfix.rst @@ -0,0 +1 @@ +Fixed :meth:`~aiohttp.StreamReader.readany` and :meth:`~aiohttp.StreamReader.read_nowait` joining data fed back into the buffer during the call (when draining below the low water mark resumes reading) into a single unbounded :class:`bytes`; a call now returns only the chunks that were buffered when it started, keeping the drain of an unread auto-decompressed request body bounded by the read buffer -- by :user:`bdraco`. diff --git a/aiohttp/streams.py b/aiohttp/streams.py index 196469c005a..e1a5b531470 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -560,14 +560,21 @@ def _read_nowait(self, n: int) -> bytes: """Read not more than n bytes, or whole buffer if n == -1""" self._timer.assert_timeout() - chunks = [] + if n == -1: + # Drain only chunks present now; _read_nowait_chunk() can + # re-entrantly resume_reading() and refill the buffer. + count = len(self._buffer) + if count == 1: + return self._read_nowait_chunk(-1) + return b"".join([self._read_nowait_chunk(-1) for _ in range(count)]) + + chunks: list[bytes] = [] while self._buffer: chunk = self._read_nowait_chunk(n) chunks.append(chunk) - if n != -1: - n -= len(chunk) - if n == 0: - break + n -= len(chunk) + if n == 0: + break return b"".join(chunks) if chunks else b"" diff --git a/tests/test_streams.py b/tests/test_streams.py index ae3c731e477..8c43951ab86 100644 --- a/tests/test_streams.py +++ b/tests/test_streams.py @@ -1723,3 +1723,31 @@ def resume_reading() -> None: protocol.resume_reading.assert_called() assert protocol._reading_paused is False + + +async def test_readany_does_not_drain_reentrant_refill( + protocol: mock.Mock, +) -> None: + """A single readany() must not reassemble data fed re-entrantly. + + Draining below the low water mark resumes reading, which can synchronously + refill the buffer (e.g. decompressing another chunk). Joining that refill in + one call would reassemble an unbounded body. + """ + loop = asyncio.get_running_loop() + stream = streams.StreamReader(protocol, limit=4, loop=loop) + + refills = [b"second", b"third"] + + def resume_reading() -> None: + if refills: + stream.feed_data(refills.pop(0)) + + protocol.resume_reading.side_effect = resume_reading + + stream.feed_data(b"first") + + # Popping "first" refills "second", but this readany() returns only "first". + assert await stream.readany() == b"first" + assert await stream.readany() == b"second" + assert await stream.readany() == b"third" diff --git a/tests/test_web_functional.py b/tests/test_web_functional.py index f054f71db21..231a79c2757 100644 --- a/tests/test_web_functional.py +++ b/tests/test_web_functional.py @@ -4,7 +4,9 @@ import pathlib import socket import sys +import zlib from collections.abc import Generator +from contextlib import suppress from typing import Any, NoReturn from unittest import mock @@ -24,7 +26,9 @@ ) from aiohttp.compression_utils import ZLibBackend, ZLibCompressObjProtocol from aiohttp.hdrs import CONTENT_LENGTH, CONTENT_TYPE, TRANSFER_ENCODING +from aiohttp.helpers import DEFAULT_CHUNK_SIZE from aiohttp.pytest_plugin import AiohttpClient, AiohttpServer +from aiohttp.streams import StreamReader from aiohttp.typedefs import Handler from aiohttp.web_protocol import RequestHandler @@ -1711,6 +1715,65 @@ async def handler(request): await resp.release() +@pytest.mark.parametrize("decompressed_size", [4 * 1024 * 1024, 32 * 1024 * 1024]) +async def test_unread_compressed_body_drain_is_bounded( + aiohttp_server: AiohttpServer, + monkeypatch: pytest.MonkeyPatch, + decompressed_size: int, +) -> None: + """Draining an unread compressed body stays bounded by the read buffer. + + A handler that rejects before reading still drains the payload during + lingering close; a small compressed body must not force a large transient + allocation (a deflate-bomb style DoS). + """ + drain_reads: list[int] = [] + drained = asyncio.Event() + readany = StreamReader.readany + + async def record_readany(self: StreamReader) -> bytes: + data = await readany(self) + assert data + drain_reads.append(len(data)) + drained.set() + return data + + monkeypatch.setattr(StreamReader, "readany", record_readany) + + async def handler(request: web.Request) -> web.Response: + return web.Response(status=401) + + app = web.Application(client_max_size=1024) + app.router.add_post("/", handler) + server = await aiohttp_server(app) + + body = zlib.compress(b"a" * decompressed_size) + assert len(body) < decompressed_size + head = ( + b"POST / HTTP/1.1\r\n" + b"Host: localhost\r\n" + b"Content-Encoding: deflate\r\n" + b"Content-Length: %d\r\n" + b"Connection: keep-alive\r\n\r\n" + ) % len(body) + + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write(head + body) + await writer.drain() + status_line = await asyncio.wait_for(reader.readline(), 5) + assert status_line.startswith(b"HTTP/1.1 401 ") + await asyncio.wait_for(drained.wait(), 5) + finally: + writer.close() + with suppress(ConnectionResetError, BrokenPipeError): + await writer.wait_closed() + + # Bounded by the buffer, not the decompressed size. + assert max(drain_reads) <= 3 * DEFAULT_CHUNK_SIZE + assert max(drain_reads) < decompressed_size + + async def test_app_max_client_size(aiohttp_client) -> None: async def handler(request): await request.post() From a329a7aacad5284f087af36103aff778746da0f2 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:40:24 -0500 Subject: [PATCH 37/49] [PR #12824/60b85e98 backport][3.14] Preserve host-only cookie scope across CookieJar save/load (#12833) --- CHANGES/12824.bugfix.rst | 1 + aiohttp/cookiejar.py | 52 ++++++++++----- tests/test_cookiejar.py | 135 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 171 insertions(+), 17 deletions(-) create mode 100644 CHANGES/12824.bugfix.rst diff --git a/CHANGES/12824.bugfix.rst b/CHANGES/12824.bugfix.rst new file mode 100644 index 00000000000..f8dbd169c31 --- /dev/null +++ b/CHANGES/12824.bugfix.rst @@ -0,0 +1 @@ +Fixed :class:`~aiohttp.CookieJar` dropping the host-only flag of cookies when persisted with :meth:`~aiohttp.CookieJar.save` and reloaded with :meth:`~aiohttp.CookieJar.load`, so a cookie set without a ``Domain`` attribute is again scoped to the exact host that set it after a reload; the absolute expiration deadline is now persisted as well, so a reloaded cookie keeps its original lifetime instead of being rescheduled from the load time. :meth:`~aiohttp.CookieJar.load` now replaces the jar contents rather than merging onto prior state, and loaded cookies pass through the same acceptance rules as :meth:`~aiohttp.CookieJar.update_cookies`, so a cookie for an IP-address host is dropped when loaded into a jar created without ``unsafe=True`` -- by :user:`bdraco`. diff --git a/aiohttp/cookiejar.py b/aiohttp/cookiejar.py index e1579c0ed4c..2b972247009 100644 --- a/aiohttp/cookiejar.py +++ b/aiohttp/cookiejar.py @@ -39,6 +39,9 @@ _MIN_SCHEDULED_COOKIE_EXPIRATION = 100 _SIMPLE_COOKIE = SimpleCookie() +# Not persisted; the absolute deadline is saved instead. +_RELATIVE_EXPIRY_ATTRS = frozenset(("max-age", "expires")) + class _RestrictedCookieUnpickler(pickle._Unpickler): """A restricted unpickler that only allows cookie-related types. @@ -174,21 +177,28 @@ def save(self, file_path: PathLike) -> None: :class:`str` or :class:`pathlib.Path` instance. """ file_path = pathlib.Path(file_path) - data: dict[str, dict[str, dict[str, str | bool]]] = {} + data: dict[str, dict[str, dict[str, str | bool | float]]] = {} for (domain, path), cookie in self._cookies.items(): key = f"{domain}|{path}" data[key] = {} for name, morsel in cookie.items(): - morsel_data: dict[str, str | bool] = { + morsel_data: dict[str, str | bool | float] = { "key": morsel.key, "value": morsel.value, "coded_value": morsel.coded_value, } - # Save all morsel attributes that have values + # Skip relative expiry; the absolute deadline is saved below. for attr in morsel._reserved: # type: ignore[attr-defined] + if attr in _RELATIVE_EXPIRY_ATTRS: + continue attr_val = morsel[attr] if attr_val: morsel_data[attr] = attr_val + # Persist or it reloads as a domain cookie and leaks to subdomains. + if (domain, name) in self._host_only_cookies: + morsel_data["host_only"] = True + if (exp := self._expirations.get((domain, path, name))) is not None: + morsel_data["expires_timestamp"] = exp data[key][name] = morsel_data # Cookie persistence may include authentication/session tokens. @@ -209,6 +219,9 @@ def load(self, file_path: PathLike) -> None: pickle format (using a restricted unpickler) for backward compatibility with existing cookie files. + Replaces the current jar contents; loaded cookies pass through the + same acceptance rules as :meth:`update_cookies`. + :param file_path: Path to file from where cookies will be imported, :class:`str` or :class:`pathlib.Path` instance. """ @@ -217,32 +230,28 @@ def load(self, file_path: PathLike) -> None: try: with file_path.open(mode="r", encoding="utf-8") as f: data = json.load(f) - self._cookies = self._load_json_data(data) + self._load_json_data(data) except (json.JSONDecodeError, UnicodeDecodeError, ValueError): # Fall back to legacy pickle format with restricted unpickler with file_path.open(mode="rb") as f: self._cookies = _RestrictedCookieUnpickler(f).load() def _load_json_data( - self, data: dict[str, dict[str, dict[str, str | bool]]] - ) -> defaultdict[tuple[str, str], SimpleCookie]: - """Load cookies from parsed JSON data.""" - cookies: defaultdict[tuple[str, str], SimpleCookie] = defaultdict(SimpleCookie) + self, data: dict[str, dict[str, dict[str, str | bool | float]]] + ) -> None: + """Replace contents, routing cookies through update_cookies().""" + self.clear() for compound_key, cookie_data in data.items(): domain, path = compound_key.split("|", 1) - key = (domain, path) for name, morsel_data in cookie_data.items(): morsel: Morsel[str] = Morsel() - morsel_key = morsel_data["key"] - morsel_value = morsel_data["value"] - morsel_coded_value = morsel_data["coded_value"] # Use __setstate__ to bypass validation, same pattern # used in _build_morsel and _cookie_helpers. morsel.__setstate__( # type: ignore[attr-defined] { - "key": morsel_key, - "value": morsel_value, - "coded_value": morsel_coded_value, + "key": morsel_data["key"], + "value": morsel_data["value"], + "coded_value": morsel_data["coded_value"], } ) # Restore morsel attributes @@ -253,8 +262,17 @@ def _load_json_data( "coded_value", ): morsel[attr] = morsel_data[attr] - cookies[key][name] = morsel - return cookies + # Drop the domain so update_cookies() re-marks it host-only. + if morsel_data.get("host_only"): + morsel["domain"] = "" + response_url = ( + URL.build(scheme="https", host=domain) if domain else URL() + ) + self.update_cookies({name: morsel}, response_url) + # Restore the absolute deadline; update_cookies() schedules none. + if (exp := morsel_data.get("expires_timestamp")) is not None: + self._expire_cookie(float(exp), domain, path, name) + self._do_expiration() def clear(self, predicate: ClearCookiePredicate | None = None) -> None: if predicate is None: diff --git a/tests/test_cookiejar.py b/tests/test_cookiejar.py index fa1e31cd9f6..9276ff252dc 100644 --- a/tests/test_cookiejar.py +++ b/tests/test_cookiejar.py @@ -2,6 +2,7 @@ import datetime import heapq import itertools +import json import logging import os import pathlib @@ -1814,6 +1815,140 @@ async def test_save_load_json_partitioned_cookies(tmp_path: Path) -> None: assert s["path"] == lo["path"] +async def test_save_load_json_preserves_host_only_scope(tmp_path: Path) -> None: + """Verify save/load keeps host-only cookies off subdomains.""" + file_path = tmp_path / "host_only.json" + issuer = URL("https://auth.example.com/login") + subdomain = URL("https://sub.auth.example.com/") + + jar_save = CookieJar() + jar_save.update_cookies({"sid": "hostonly"}, response_url=issuer) + assert "sid" not in jar_save.filter_cookies(subdomain) + jar_save.save(file_path=file_path) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + assert jar_load.host_only_cookies == frozenset({("auth.example.com", "sid")}) + assert "sid" not in jar_load.filter_cookies(subdomain) + assert "sid" in jar_load.filter_cookies(issuer) + + +async def test_save_load_json_domain_cookie_still_matches_subdomain( + tmp_path: Path, +) -> None: + """Verify save/load keeps an explicit Domain cookie valid for subdomains.""" + file_path = tmp_path / "domain.json" + subdomain = URL("https://sub.example.com/") + + jar_save = CookieJar() + jar_save.update_cookies_from_headers( + ["sid=domaincookie; Domain=example.com"], URL("https://example.com/") + ) + jar_save.save(file_path=file_path) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + assert jar_load.host_only_cookies == frozenset() + assert "sid" in jar_load.filter_cookies(subdomain) + + +async def test_save_load_json_preserves_max_age_deadline(tmp_path: Path) -> None: + """Verify save/load restores the absolute deadline without resetting it.""" + file_path = tmp_path / "max_age.json" + url = URL("https://example.com/") + + jar_save = CookieJar() + jar_save.update_cookies_from_headers( + ["sid=x; Max-Age=3600; Domain=example.com"], url + ) + expirations = dict(jar_save._expirations) + jar_save.save(file_path=file_path) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + # The deadline is restored as the original absolute time, not now + Max-Age. + assert dict(jar_load._expirations) == expirations + assert "sid" in jar_load.filter_cookies(url) + + +async def test_save_load_json_drops_expired_cookie(tmp_path: Path) -> None: + """Verify a cookie whose persisted deadline is in the past is dropped on load.""" + file_path = tmp_path / "expired.json" + url = URL("https://example.com/") + + # Save a future-expiring cookie, then rewrite its persisted deadline to the + # past so the cookie survives save() and the drop happens on the load path. + jar_save = CookieJar() + jar_save.update_cookies_from_headers( + ["sid=x; Expires=Tue, 1 Jan 2999 12:00:00 GMT; Domain=example.com"], url + ) + jar_save.save(file_path=file_path) + data = json.loads(file_path.read_text()) + _, cookies = next(iter(data.items())) + cookies["sid"]["expires_timestamp"] = 0.0 + file_path.write_text(json.dumps(data)) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + assert len(jar_load) == 0 + assert "sid" not in jar_load.filter_cookies(url) + + +async def test_save_load_json_preserves_expires_deadline(tmp_path: Path) -> None: + """Verify a future Expires deadline survives a save/load roundtrip.""" + file_path = tmp_path / "expires.json" + url = URL("https://example.com/") + + jar_save = CookieJar() + jar_save.update_cookies_from_headers( + ["sid=x; Expires=Tue, 1 Jan 2999 12:00:00 GMT; Domain=example.com"], url + ) + expirations = dict(jar_save._expirations) + jar_save.save(file_path=file_path) + + jar_load = CookieJar() + jar_load.load(file_path=file_path) + + assert dict(jar_load._expirations) == expirations + assert "sid" in jar_load.filter_cookies(url) + + +async def test_load_json_old_format_without_new_keys(tmp_path: Path) -> None: + """Verify a file written by an older version (no host_only/expires_timestamp) loads.""" + file_path = tmp_path / "old.json" + # Old schema: no host_only, no expires_timestamp; relative max-age morsel attr. + file_path.write_text( + json.dumps( + { + "example.com|/": { + "sid": { + "key": "sid", + "value": "x", + "coded_value": "x", + "domain": "example.com", + "max-age": "3600", + } + } + } + ) + ) + url = URL("https://example.com/") + + jar_load = CookieJar() + # No exception when the new keys are absent. + jar_load.load(file_path=file_path) + + # A host-only cookie saved without Domain by an older version had no domain + # field, so it now loads as a domain cookie (the documented migration loss). + assert "sid" in jar_load.filter_cookies(url) + # max-age is rescheduled from load time rather than an absolute deadline. + assert any(key[2] == "sid" for key in jar_load._expirations) + + async def test_json_format_is_safe(tmp_path: Path) -> None: """Verify the JSON file format cannot execute code on load.""" import json From a762eda5242f6490d6ba667533193f8b473ad587 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 05:47:16 +0000 Subject: [PATCH 38/49] [PR #12831/1ac92dae backport][3.14] Payload close on disconnect (#12843) Co-authored-by: J. Nick Koston --- CHANGES/12831.bugfix.rst | 1 + aiohttp/web_response.py | 6 ++-- tests/test_web_response.py | 73 +++++++++++++++++++++++++++++++++++++- 3 files changed, 77 insertions(+), 3 deletions(-) create mode 100644 CHANGES/12831.bugfix.rst diff --git a/CHANGES/12831.bugfix.rst b/CHANGES/12831.bugfix.rst new file mode 100644 index 00000000000..bf460ffccac --- /dev/null +++ b/CHANGES/12831.bugfix.rst @@ -0,0 +1 @@ +Fixed :meth:`aiohttp.web.Response.write_eof` skipping ``Payload.close()`` when the body write was interrupted by an error or cancellation, for example when a client disconnects mid-response; the payload close hook now runs in a ``finally`` so a :class:`~aiohttp.payload.Payload` body always releases its resources -- by :user:`bdraco`. diff --git a/aiohttp/web_response.py b/aiohttp/web_response.py index daf29eccce4..cbe4985cfb9 100644 --- a/aiohttp/web_response.py +++ b/aiohttp/web_response.py @@ -808,8 +808,10 @@ async def write_eof(self, data: bytes = b"") -> None: if body is None or self._must_be_empty_body: await super().write_eof() elif isinstance(self._body, Payload): - await self._body.write(self._payload_writer) - await self._body.close() + try: + await self._body.write(self._payload_writer) + finally: + await self._body.close() await super().write_eof() else: await super().write_eof(cast(bytes, body)) diff --git a/tests/test_web_response.py b/tests/test_web_response.py index 52a574f1c0c..c5ca849b6b9 100644 --- a/tests/test_web_response.py +++ b/tests/test_web_response.py @@ -1,3 +1,4 @@ +import asyncio import collections.abc import datetime import gzip @@ -17,7 +18,7 @@ from aiohttp.helpers import ETag from aiohttp.http_writer import StreamWriter, _serialize_headers from aiohttp.multipart import BodyPartReader, MultipartWriter -from aiohttp.payload import BytesPayload, StringPayload +from aiohttp.payload import BytesPayload, Payload, StringPayload from aiohttp.test_utils import make_mocked_request from aiohttp.web import ( ContentCoding, @@ -1434,6 +1435,76 @@ async def test_consecutive_write_eof() -> None: writer.write_eof.assert_called_once_with(data) +class _ClosingPayload(Payload): + """Payload test double that records whether close() ran.""" + + def __init__(self) -> None: + super().__init__(None) + self.close_called = False + self.started = asyncio.Event() + self.release = asyncio.Event() + self.fail = False + + async def write(self, writer: AbstractStreamWriter) -> None: + self.started.set() + if self.fail: + raise ConnectionResetError("client gone") + await self.release.wait() + + async def close(self) -> None: + self.close_called = True + await super().close() + + def decode(self, encoding: str = "utf-8", errors: str = "strict") -> str: + assert False + + +async def test_write_eof_closes_payload_on_success() -> None: + writer = mock.create_autospec(AbstractStreamWriter, spec_set=True, instance=True) + req = make_request("GET", "/", writer=writer) + payload = _ClosingPayload() + payload.release.set() + resp = web.Response(body=payload) + + await resp.prepare(req) + await resp.write_eof() + + assert payload.close_called + assert writer.write_eof.called + + +async def test_write_eof_closes_payload_on_write_error() -> None: + writer = mock.create_autospec(AbstractStreamWriter, spec_set=True, instance=True) + req = make_request("GET", "/", writer=writer) + payload = _ClosingPayload() + payload.fail = True + resp = web.Response(body=payload) + + await resp.prepare(req) + with pytest.raises(ConnectionResetError): + await resp.write_eof() + + assert payload.close_called + assert not writer.write_eof.called + + +async def test_write_eof_closes_payload_on_cancel() -> None: + writer = mock.create_autospec(AbstractStreamWriter, spec_set=True, instance=True) + req = make_request("GET", "/", writer=writer) + payload = _ClosingPayload() + resp = web.Response(body=payload) + + await resp.prepare(req) + task = asyncio.ensure_future(resp.write_eof()) + await payload.started.wait() + task.cancel() + with pytest.raises(asyncio.CancelledError): + await task + + assert payload.close_called + assert not writer.write_eof.called + + def test_set_text_with_content_type() -> None: resp = Response() resp.content_type = "text/html" From 0e9cedd995c6ebaa84ae1a9148212599e0e888f4 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 00:50:43 -0500 Subject: [PATCH 39/49] [PR #12827/ccf218ab backport][3.14] Numeric ipv4 resolver bypass (#12849) --- CHANGES/12827.bugfix.rst | 1 + aiohttp/connector.py | 7 ++++ aiohttp/helpers.py | 22 ++++++++++ tests/test_connector.py | 30 ++++++++++++++ tests/test_helpers.py | 89 ++++++++++++++++++++++++++++++++++++++++ 5 files changed, 149 insertions(+) create mode 100644 CHANGES/12827.bugfix.rst diff --git a/CHANGES/12827.bugfix.rst b/CHANGES/12827.bugfix.rst new file mode 100644 index 00000000000..9442867d363 --- /dev/null +++ b/CHANGES/12827.bugfix.rst @@ -0,0 +1 @@ +Changed :class:`~aiohttp.TCPConnector` to reject legacy non-canonical numeric IPv4 host forms such as ``2130706433``, ``017700000001`` and ``127.1`` with :exc:`~aiohttp.InvalidUrlClientError`; only canonical dotted-quad IPv4 literals are now treated as IP address literals, while every other host is sent through the configured resolver -- by :user:`bdraco`. diff --git a/aiohttp/connector.py b/aiohttp/connector.py index b206f2e989a..b7aa7b7647a 100644 --- a/aiohttp/connector.py +++ b/aiohttp/connector.py @@ -27,6 +27,7 @@ ClientConnectorSSLError, ClientHttpProxyError, ClientProxyConnectionError, + InvalidUrlClientError, ServerFingerprintMismatch, UnixClientConnectorError, cert_errors, @@ -37,6 +38,7 @@ from .helpers import ( _SENTINEL, ceil_timeout, + is_canonical_ipv4_address, is_ip_address, noop, sentinel, @@ -1092,6 +1094,11 @@ async def _resolve_host( ) -> list[ResolveResult]: """Resolve host and return list of addresses.""" if is_ip_address(host): + # Reject legacy numeric IPv4 forms (e.g. 2130706433, 127.1) that + # socket would map onto an address, slipping past a connector-level + # policy that only sees the raw host. + if ":" not in host and not is_canonical_ipv4_address(host): + raise InvalidUrlClientError(host, "is not a canonical IPv4 address") return [ { "hostname": host, diff --git a/aiohttp/helpers.py b/aiohttp/helpers.py index 5988aab4aa8..469c99dd63c 100644 --- a/aiohttp/helpers.py +++ b/aiohttp/helpers.py @@ -511,6 +511,28 @@ def is_ip_address(host: str | None) -> bool: return ":" in host or host.replace(".", "").isdigit() +def is_canonical_ipv4_address(host: str) -> bool: + """Check if host is a canonical dotted-quad IPv4 address. + + Rejects the legacy numeric forms that ``socket`` still accepts and + maps onto an address, e.g. ``2130706433``, ``017700000001``, ``127.1``. + """ + parts = host.split(".") + if len(parts) != 4: + return False + for part in parts: + # Each octet must be 1-3 ASCII digits; reject unicode digits + # (which ``str.isdigit`` accepts but ``int`` may not), octal + # leading zeros, and values above 255. + if not (1 <= len(part) <= 3) or not part.isascii() or not part.isdigit(): + return False + if part[0] == "0" and len(part) != 1: + return False + if int(part) > 255: + return False + return True + + _cached_current_datetime: int | None = None _cached_formatted_datetime = "" diff --git a/tests/test_connector.py b/tests/test_connector.py index 27a7cc43ff6..75bd1461ba1 100644 --- a/tests/test_connector.py +++ b/tests/test_connector.py @@ -26,6 +26,7 @@ from aiohttp import client, connector as connector_module, hdrs, web from aiohttp.abc import AbstractResolver from aiohttp.client import ClientRequest, ClientTimeout +from aiohttp.client_exceptions import InvalidUrlClientError from aiohttp.client_proto import ResponseHandler from aiohttp.client_reqrep import ConnectionKey from aiohttp.connector import ( @@ -1253,6 +1254,35 @@ async def test_tcp_connector_resolve_host(loop: asyncio.AbstractEventLoop) -> No await conn.close() +async def test_tcp_connector_rejects_non_canonical_ipv4_alias() -> None: + """Legacy numeric IPv4 aliases must not bypass the configured resolver.""" + calls: list[str] = [] + + class _RecordingResolver(AbstractResolver): + async def resolve( + self, + host: str, + port: int = 0, + family: socket.AddressFamily = socket.AF_INET, + ) -> list[ResolveResult]: + assert False + + async def close(self) -> None: + """Close the resolver.""" + + conn = aiohttp.TCPConnector(resolver=_RecordingResolver()) + for alias in ("2130706433", "017700000001", "127.1"): + with pytest.raises(InvalidUrlClientError, match="canonical IPv4"): + await conn._resolve_host(alias, 8080) + + # Resolver is never consulted, and a canonical IP still short-circuits it. + assert calls == [] + res = await conn._resolve_host("127.0.0.1", 8080) + assert res[0]["host"] == "127.0.0.1" + assert calls == [] + await conn.close() + + @pytest.fixture def dns_response(loop): async def coro(): diff --git a/tests/test_helpers.py b/tests/test_helpers.py index 3ddf79c7666..081b807cca4 100644 --- a/tests/test_helpers.py +++ b/tests/test_helpers.py @@ -2,6 +2,8 @@ import base64 import datetime import gc +import ipaddress +import itertools import sys import warnings import weakref @@ -319,6 +321,93 @@ def test_is_ip_address_invalid_type() -> None: helpers.is_ip_address(object()) +# ------------------------------- is_canonical_ipv4_address() --------------- + + +@pytest.mark.parametrize( + "host", + [ + "0.0.0.0", + "127.0.0.1", + "8.8.8.8", + "192.168.0.1", + "255.255.255.255", + ], +) +def test_is_canonical_ipv4_address_accepts_dotted_quad(host: str) -> None: + assert helpers.is_canonical_ipv4_address(host) + + +@pytest.mark.parametrize( + "host", + [ + "2130706433", # decimal integer form of 127.0.0.1 + "017700000001", # octal form of 127.0.0.1 + "127.1", # short-hand form of 127.0.0.1 + "127.0.1", # 3-part short-hand + "0177.0.0.1", # octal leading-zero octet + "01.2.3.4", # octal leading-zero octet + "256.0.0.1", # octet out of range + "999.0.0.1", # octet out of range + "1.2.3.4.5", # too many octets + "127.0.0.", # trailing dot / empty octet + "12³.0.0.1", # superscript digit (str.isdigit but not int) + "１２７.0.0.1", # full-width digits + "0xa.0.0.0", # hex octet + " 127.0.0.1", # leading whitespace + "127.0.0.1 ", # trailing whitespace + "example.com", # domain name + "", # empty + ], +) +def test_is_canonical_ipv4_address_rejects_non_canonical(host: str) -> None: + assert not helpers.is_canonical_ipv4_address(host) + + +def _ipaddress_accepts_ipv4(host: str) -> bool: + """Oracle: does the stdlib accept ``host`` as a canonical IPv4 address?""" + try: + ipaddress.IPv4Address(host) + except ipaddress.AddressValueError: + return False + return True + + +def test_is_canonical_ipv4_address_matches_stdlib() -> None: + """Prove equivalence with ``ipaddress.IPv4Address`` over a broad corpus. + + The helper is a fast hand-rolled substitute for the stdlib parser; this + exhaustively cross-checks the two agree on every combination of a set of + octet-like tokens covering the known edge cases (leading zeros, out of + range, empty, unicode digits, wrong octet count). + """ + tokens = [ + "0", + "1", + "9", + "10", + "99", + "255", + "256", + "999", + "00", + "01", + "0177", + "1234", + "", + "a", + "0x1", + "１", # full-width 1 + "1²", # trailing superscript + ] + for count in range(1, 5): + for parts in itertools.product(tokens, repeat=count): + host = ".".join(parts) + assert helpers.is_canonical_ipv4_address(host) == _ipaddress_accepts_ipv4( + host + ), host + + # ----------------------------------- TimeoutHandle ------------------- From d8db2e7e22140b694ad29aa19b6ac131df091fb6 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 01:11:38 -0500 Subject: [PATCH 40/49] [PR #12830/93a2b1c3 backport][3.15] Bound pipelined request queue per connection (#12855) --- CHANGES/12830.bugfix.rst | 1 + aiohttp/_http_parser.pyx | 20 +++++- aiohttp/base_protocol.py | 25 ++++++- aiohttp/http_parser.py | 20 ++++++ aiohttp/web_protocol.py | 68 +++++++++++++++++- docs/spelling_wordlist.txt | 1 + tests/test_http_parser.py | 72 +++++++++++++++++++ tests/test_web_functional.py | 132 ++++++++++++++++++++++++++++++++++- tests/test_web_protocol.py | 93 ++++++++++++++++++++++++ 9 files changed, 425 insertions(+), 7 deletions(-) create mode 100644 CHANGES/12830.bugfix.rst diff --git a/CHANGES/12830.bugfix.rst b/CHANGES/12830.bugfix.rst new file mode 100644 index 00000000000..d44d76da404 --- /dev/null +++ b/CHANGES/12830.bugfix.rst @@ -0,0 +1 @@ +Bounded the number of parsed-but-unhandled pipelined HTTP/1 requests buffered per connection on the server; once the queue reaches an internal limit the parser stops emitting and the transport is paused, resuming as the request handler drains the queue, so a client keeping one handler busy can no longer accumulate an unbounded backlog of pipelined requests -- by :user:`bdraco`. diff --git a/aiohttp/_http_parser.pyx b/aiohttp/_http_parser.pyx index b5e5d6fc375..5ca1ccf6b0d 100644 --- a/aiohttp/_http_parser.pyx +++ b/aiohttp/_http_parser.pyx @@ -323,6 +323,8 @@ cdef class HttpParser: list _messages bint _more_data_available bint _paused + Py_ssize_t _msg_in_flight + Py_ssize_t _max_msg_queue_size bint _eof_pending object _payload unsigned long long _content_length_expected @@ -359,6 +361,7 @@ cdef class HttpParser: size_t max_field_size=8190, payload_exception=None, bint response_with_body=True, bint read_until_eof=False, bint auto_decompress=True, + Py_ssize_t max_msg_queue_size=0, ): cparser.llhttp_settings_init(self._csettings) cparser.llhttp_init(self._cparser, mode, self._csettings) @@ -373,6 +376,8 @@ cdef class HttpParser: self._buf = bytearray() self._more_data_available = False self._paused = False + self._msg_in_flight = 0 + self._max_msg_queue_size = max_msg_queue_size self._eof_pending = False self._payload = None self._payload_error = 0 @@ -556,6 +561,11 @@ cdef class HttpParser: assert self._payload is not None self._paused = True + def message_consumed(self): + # Protocol drained a queued message; free a slot for parsing. + if self._msg_in_flight > 0: + self._msg_in_flight -= 1 + def feed_eof(self): cdef bytes desc @@ -678,12 +688,12 @@ cdef class HttpRequestParser(HttpParser): size_t max_line_size=8190, size_t max_headers=128, size_t max_field_size=8190, payload_exception=None, bint response_with_body=True, bint read_until_eof=False, - bint auto_decompress=True, + bint auto_decompress=True, Py_ssize_t max_msg_queue_size=0, ): self._init(cparser.HTTP_REQUEST, protocol, loop, limit, timer, max_line_size, max_headers, max_field_size, payload_exception, response_with_body, read_until_eof, - auto_decompress) + auto_decompress, max_msg_queue_size) cdef object _on_status_complete(self): cdef int idx1, idx2 @@ -893,6 +903,12 @@ cdef int cb_on_message_complete(cparser.llhttp_t* parser) except -1: pyparser._last_error = exc return -1 else: + if pyparser._max_msg_queue_size: + pyparser._msg_in_flight += 1 + if pyparser._msg_in_flight >= pyparser._max_msg_queue_size: + # Queue full: pause llhttp between messages. feed_data() buffers + # the remainder as tail; resumes once the queue drains. + return cparser.HPE_PAUSED return 0 diff --git a/aiohttp/base_protocol.py b/aiohttp/base_protocol.py index f1f6edc3836..df3f8c089ac 100644 --- a/aiohttp/base_protocol.py +++ b/aiohttp/base_protocol.py @@ -8,6 +8,13 @@ if TYPE_CHECKING: from .http_parser import HttpParser +# Raised by transport.pause_reading()/resume_reading() when the transport +# does not support flow control; safe to ignore. +# NOTE: Catch these with a plain try/except/pass, never contextlib.suppress(): +# pause/resume run on the hot read path and suppress() is ~6x slower than +# try/except here (it builds a context manager and unpacks this tuple per call). +PAUSE_RESUME_READING_ERRORS = (AttributeError, NotImplementedError, RuntimeError) + class BaseProtocol(asyncio.Protocol): __slots__ = ( @@ -65,9 +72,15 @@ def pause_reading(self) -> None: if self.transport is not None: try: self.transport.pause_reading() - except (AttributeError, NotImplementedError, RuntimeError): + except PAUSE_RESUME_READING_ERRORS: + # Transport lacks flow control; nothing to pause. Intentionally + # ignored (see PAUSE_RESUME_READING_ERRORS; do not use suppress). pass + def _reading_paused_for_msg_queue(self) -> bool: + """Keep the transport paused for protocol-specific reasons (overridden).""" + return False + def resume_reading(self, resume_parser: bool = True) -> None: self._reading_paused = False @@ -77,10 +90,16 @@ def resume_reading(self, resume_parser: bool = True) -> None: # Reading may have been paused again in the above call if there was a lot of # compressed data still pending. - if not self._reading_paused and self.transport is not None: + if ( + not self._reading_paused + and not self._reading_paused_for_msg_queue() + and self.transport is not None + ): try: self.transport.resume_reading() - except (AttributeError, NotImplementedError, RuntimeError): + except PAUSE_RESUME_READING_ERRORS: + # Transport lacks flow control; nothing to resume. Intentionally + # ignored (see PAUSE_RESUME_READING_ERRORS; do not use suppress). pass self._reading_paused = False diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py index 5d48dedaa6c..5b46101ddaa 100644 --- a/aiohttp/http_parser.py +++ b/aiohttp/http_parser.py @@ -275,6 +275,7 @@ def __init__( response_with_body: bool = True, read_until_eof: bool = False, auto_decompress: bool = True, + max_msg_queue_size: int = 0, ) -> None: self.protocol = protocol self.loop = loop @@ -300,6 +301,9 @@ def __init__( self._headers_parser = HeadersParser( max_line_size, max_headers, max_field_size, self.lax ) + # Stop emitting messages once this many are queued unconsumed (0 = off). + self._max_msg_queue_size = max_msg_queue_size + self._msg_in_flight = 0 @abc.abstractmethod def parse_message(self, lines: list[bytes]) -> _MsgT: ... @@ -311,6 +315,11 @@ def pause_reading(self) -> None: assert self._payload_parser is not None self._payload_parser.pause_reading() + def message_consumed(self) -> None: + """Protocol drained a queued message; free a slot for parsing.""" + if self._msg_in_flight > 0: + self._msg_in_flight -= 1 + def feed_eof(self) -> _MsgT | None: if self._payload_parser is not None: self._payload_parser.feed_eof() @@ -353,6 +362,15 @@ def feed_data( # read HTTP message (request/response line + headers), \r\n\r\n # and split by lines if self._payload_parser is None and not self._upgraded: + if ( + self._max_msg_queue_size + and self._msg_in_flight >= self._max_msg_queue_size + ): + # Queue full: buffer the rest and stop. Safe pause point; + # any preceding body is consumed before the next request + # line. Resumes via feed_data(b"") when the queue drains. + self._tail = data[start_pos:] + break pos = data.find(SEP, start_pos) # consume \r\n if pos == start_pos and not self._lines: @@ -497,6 +515,8 @@ def get_content_length() -> int | None: payload = EMPTY_PAYLOAD messages.append((msg, payload)) + if self._max_msg_queue_size: + self._msg_in_flight += 1 should_close = msg.should_close else: self._tail = data[start_pos:] diff --git a/aiohttp/web_protocol.py b/aiohttp/web_protocol.py index d0719a7481d..f1b384434e0 100644 --- a/aiohttp/web_protocol.py +++ b/aiohttp/web_protocol.py @@ -16,7 +16,7 @@ from propcache import under_cached_property from .abc import AbstractAccessLogger, AbstractStreamWriter -from .base_protocol import BaseProtocol +from .base_protocol import PAUSE_RESUME_READING_ERRORS, BaseProtocol from .helpers import DEFAULT_CHUNK_SIZE, ceil_timeout from .http import ( HttpProcessingError, @@ -37,6 +37,11 @@ __all__ = ("RequestHandler", "RequestPayloadError", "PayloadAccessError") +# Max parsed-but-unhandled pipelined requests buffered per connection before +# reading is paused. Bounds memory a client can pin by keeping one handler busy +# and pipelining behind it; reading resumes as the queue drains. +MAX_MSG_QUEUE_SIZE = 32 + if TYPE_CHECKING: import ssl @@ -146,6 +151,9 @@ class RequestHandler(BaseProtocol): "_keepalive_timeout", "_lingering_time", "_messages", + "_max_msg_queue_size", + "_msg_queue_resume_size", + "_msg_queue_paused", "_message_tail", "_handler_waiter", "_waiter", @@ -186,6 +194,13 @@ def __init__( auto_decompress: bool = True, timeout_ceil_threshold: float = 5, ): + self._max_msg_queue_size = MAX_MSG_QUEUE_SIZE + # Low-water mark: resume reading once the queue drains to half the limit + # so we refill in batches instead of churning pause/resume per request. + self._msg_queue_resume_size = MAX_MSG_QUEUE_SIZE // 2 + # Set before super().__init__ so _reading_paused_for_msg_queue() is safe + # if BaseProtocol ever triggers a resume during init. + self._msg_queue_paused = False parser = HttpRequestParser( self, loop, @@ -195,6 +210,7 @@ def __init__( max_headers=max_headers, payload_exception=RequestPayloadError, auto_decompress=auto_decompress, + max_msg_queue_size=MAX_MSG_QUEUE_SIZE, ) super().__init__(loop, parser) @@ -431,6 +447,14 @@ def data_received(self, data: bytes) -> None: # don't set result twice waiter.set_result(None) + # Queue full: pause the transport (the parser already stopped + # emitting). start() resumes as it drains the queue. + if ( + not self._msg_queue_paused + and len(self._messages) >= self._max_msg_queue_size + ): + self._pause_msg_queue_reading() + self._upgraded = upgraded if upgraded and tail: self._message_tail = tail @@ -447,6 +471,36 @@ def data_received(self, data: bytes) -> None: if eof: self.close() + def _reading_paused_for_msg_queue(self) -> bool: + return self._msg_queue_paused + + def _pause_msg_queue_reading(self) -> None: + self._msg_queue_paused = True + if self.transport is not None: + try: + self.transport.pause_reading() + except PAUSE_RESUME_READING_ERRORS: + # Transport lacks flow control; nothing to pause. Intentionally + # ignored (see PAUSE_RESUME_READING_ERRORS; do not use suppress). + pass + + def _resume_msg_queue_reading(self) -> None: + if not self._upgraded: + # Reparse buffered pipelined requests while still marked paused so + # a refill past the limit does not re-pause an already-paused + # transport; only resume below once it stayed under the limit. + self.data_received(b"") + if len(self._messages) >= self._max_msg_queue_size: + return + self._msg_queue_paused = False + if not self._reading_paused and self.transport is not None: + try: + self.transport.resume_reading() + except PAUSE_RESUME_READING_ERRORS: + # Transport lacks flow control; nothing to resume. Intentionally + # ignored (see PAUSE_RESUME_READING_ERRORS; do not use suppress). + pass + def keep_alive(self, val: bool) -> None: """Set keep-alive connection mode. @@ -579,6 +633,18 @@ async def start(self) -> None: message, payload = self._messages.popleft() + # Free a parser slot; resume reading once drained to low water so + # pipelining keeps flowing while this request is handled. + # no branch: _parser is only None after connection_lost, whose path + # exits this loop, so the None case is not reachably exercisable. + if self._parser is not None: # pragma: no branch + self._parser.message_consumed() + if ( + self._msg_queue_paused + and len(self._messages) <= self._msg_queue_resume_size + ): + self._resume_msg_queue_reading() + # time is only fetched if logging is enabled as otherwise # its thrown away and never used. start = loop.time() if self._logging_enabled else None diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index 1de034c393b..a62ffed4ae9 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -250,6 +250,7 @@ peername performant pickleable ping +pipelined pipelining pluggable plugin diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index 9b7fa83d9a6..f2e0e9ae3f6 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -159,6 +159,78 @@ def test_c_parser_loaded(): assert "RawResponseMessageC" in dir(aiohttp.http_parser) +_PIPELINED_GET = b"GET / HTTP/1.1\r\nHost: a\r\n\r\n" + + +def _build_request_parser( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, + max_msg_queue_size: int, +) -> HttpRequestParser: + return request_cls( + protocol, + loop, + DEFAULT_CHUNK_SIZE, + max_line_size=8190, + max_headers=128, + max_field_size=8190, + max_msg_queue_size=max_msg_queue_size, + ) + + +def test_max_msg_queue_size_caps_emitted_messages( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, +) -> None: + parser = _build_request_parser(request_cls, protocol, loop, 4) + messages, upgraded, _tail = parser.feed_data(_PIPELINED_GET * 10) + assert len(messages) == 4 + assert not upgraded + + +def test_max_msg_queue_size_resumes_after_consume( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, +) -> None: + limit = 4 + total = 10 + parser = _build_request_parser(request_cls, protocol, loop, limit) + messages, _upgraded, _tail = parser.feed_data(_PIPELINED_GET * total) + seen = 0 + while messages: + assert len(messages) <= limit + seen += len(messages) + for _msg, _payload in messages: + parser.message_consumed() + messages, _upgraded, _tail = parser.feed_data(b"") + assert seen == total + + +def test_max_msg_queue_size_zero_is_unbounded( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, +) -> None: + parser = _build_request_parser(request_cls, protocol, loop, 0) + messages, _upgraded, _tail = parser.feed_data(_PIPELINED_GET * 50) + assert len(messages) == 50 + + +def test_message_consumed_underflow_is_ignored( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, +) -> None: + parser = _build_request_parser(request_cls, protocol, loop, 4) + # No message is in flight; consuming must not underflow the counter. + parser.message_consumed() + messages, _upgraded, _tail = parser.feed_data(_PIPELINED_GET * 4) + assert len(messages) == 4 + + def test_parse_headers(parser: Any) -> None: text = b"""GET /test HTTP/1.1\r Host: a\r diff --git a/tests/test_web_functional.py b/tests/test_web_functional.py index 231a79c2757..869a53797c5 100644 --- a/tests/test_web_functional.py +++ b/tests/test_web_functional.py @@ -30,7 +30,7 @@ from aiohttp.pytest_plugin import AiohttpClient, AiohttpServer from aiohttp.streams import StreamReader from aiohttp.typedefs import Handler -from aiohttp.web_protocol import RequestHandler +from aiohttp.web_protocol import MAX_MSG_QUEUE_SIZE, RequestHandler try: import brotlicffi as brotli @@ -1715,6 +1715,136 @@ async def handler(request): await resp.release() +async def test_http1_pipelined_requests_are_count_limited( + aiohttp_server: AiohttpServer, + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Requests pipelined behind a busy handler must not grow unbounded. + + A client can keep one handler active and pipeline many complete requests + behind it; the per-connection queue stays bounded by MAX_MSG_QUEUE_SIZE. + """ + pipelined_requests = 500 + slow_handler_started = asyncio.Event() + queue_observed = asyncio.Event() + max_queued = 0 + data_received = RequestHandler.data_received + + def observe_data_received(self: RequestHandler, data: bytes) -> None: + nonlocal max_queued + data_received(self, data) + if self._request_in_progress and self._messages: + max_queued = max(max_queued, len(self._messages)) + queue_observed.set() + + monkeypatch.setattr(RequestHandler, "data_received", observe_data_received) + + async def slow_handler(request: web.Request) -> web.Response: + slow_handler_started.set() + await asyncio.sleep(0.5) + return web.Response(text="slow") + + async def fast_handler(request: web.Request) -> NoReturn: + # The pipelined requests are only counted, never handled: the test + # closes the connection while the slow handler still holds the loop. + assert False + + app = web.Application() + app.router.add_get("/slow", slow_handler) + app.router.add_get("/x", fast_handler) + server = await aiohttp_server(app) + + def raw_get(path: str) -> bytes: + return ( + f"GET {path} HTTP/1.1\r\nHost: localhost\r\n" + "Connection: keep-alive\r\n\r\n" + ).encode("ascii") + + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write(raw_get("/slow")) + await writer.drain() + await asyncio.wait_for(slow_handler_started.wait(), 1) + + writer.write(raw_get("/x") * pipelined_requests) + await writer.drain() + await asyncio.wait_for(queue_observed.wait(), 1) + finally: + writer.close() + with suppress(ConnectionResetError, BrokenPipeError): + await writer.wait_closed() + + # Tight lower bound also catches over-aggressive pausing (e.g. clamping to 1). + assert MAX_MSG_QUEUE_SIZE // 2 < max_queued <= MAX_MSG_QUEUE_SIZE + + +async def test_http1_pipelined_queue_resumes_after_drain( + aiohttp_server: AiohttpServer, + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A paused pipeline queue resumes reading once handlers drain it. + + Once enough requests are pipelined behind a busy handler to fill the queue, + reading is paused; as the handlers drain the queue past the low-water mark + reading must resume so the remaining buffered requests are still served. + """ + # Several times the limit so the queue refills and re-pauses while draining. + pipelined_requests = MAX_MSG_QUEUE_SIZE * 3 + first_started = asyncio.Event() + release_first = asyncio.Event() + resumed = asyncio.Event() + handled: list[str] = [] + all_handled = asyncio.Event() + + resume = RequestHandler._resume_msg_queue_reading + + def observe_resume(self: RequestHandler) -> None: + resume(self) + resumed.set() + + monkeypatch.setattr(RequestHandler, "_resume_msg_queue_reading", observe_resume) + + async def handler(request: web.Request) -> web.Response: + if request.path == "/first": + first_started.set() + await release_first.wait() + handled.append(request.path) + if len(handled) == pipelined_requests + 1: + all_handled.set() + return web.Response() + + app = web.Application() + app.router.add_get("/{tail:.*}", handler) + server = await aiohttp_server(app) + + def raw_get(path: str) -> bytes: + return ( + f"GET {path} HTTP/1.1\r\nHost: localhost\r\n" + "Connection: keep-alive\r\n\r\n" + ).encode("ascii") + + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write(raw_get("/first")) + await writer.drain() + await asyncio.wait_for(first_started.wait(), 1) + + writer.write(b"".join(raw_get(f"/r{i}") for i in range(pipelined_requests))) + await writer.drain() + + # Let the busy handler finish so the queue drains and reading resumes. + release_first.set() + await asyncio.wait_for(resumed.wait(), 5) + # Every pipelined request is still served only if reading resumed. + await asyncio.wait_for(all_handled.wait(), 5) + finally: + writer.close() + with suppress(ConnectionResetError, BrokenPipeError): + await writer.wait_closed() + + assert len(handled) == pipelined_requests + 1 + + @pytest.mark.parametrize("decompressed_size", [4 * 1024 * 1024, 32 * 1024 * 1024]) async def test_unread_compressed_body_drain_is_bounded( aiohttp_server: AiohttpServer, diff --git a/tests/test_web_protocol.py b/tests/test_web_protocol.py index f823986b7b2..7e84356b1fe 100644 --- a/tests/test_web_protocol.py +++ b/tests/test_web_protocol.py @@ -48,3 +48,96 @@ def test_data_received_calls_data_received_cb( cb.assert_called_once() dummy_reader[1].feed_data.assert_called_once_with(b"x") + + +def test_pause_msg_queue_reading_without_transport( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """Pausing with no transport still records the paused state.""" + handler = RequestHandler(dummy_manager, loop=loop) + handler.transport = None + + handler._pause_msg_queue_reading() + + assert handler._msg_queue_paused is True + + +def test_resume_msg_queue_reading_after_upgrade_skips_reparse( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """Resume after an upgrade clears the pause and resumes without reparsing.""" + handler = RequestHandler(dummy_manager, loop=loop) + transport = mock.Mock() + handler.transport = transport + handler._upgraded = True + handler._msg_queue_paused = True + handler._reading_paused = False + + with mock.patch.object(RequestHandler, "data_received") as data_received: + handler._resume_msg_queue_reading() + + data_received.assert_not_called() + assert handler._msg_queue_paused is False + transport.resume_reading.assert_called_once_with() + + +def test_resume_msg_queue_reading_without_transport( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """Resume clears the pause but does not touch a missing transport.""" + handler = RequestHandler(dummy_manager, loop=loop) + handler.transport = None + handler._upgraded = True # skip the reparse branch + handler._msg_queue_paused = True + + handler._resume_msg_queue_reading() + + assert handler._msg_queue_paused is False + + +def test_resume_reading_stays_paused_for_msg_queue( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """Base resume_reading must not un-pause the transport while queue-paused.""" + handler = RequestHandler(dummy_manager, loop=loop) + transport = mock.Mock() + handler.transport = transport + handler._msg_queue_paused = True + + handler.resume_reading() + + transport.resume_reading.assert_not_called() + + +def test_pause_msg_queue_reading_ignores_unsupported_transport( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """A transport without flow control raising on pause is ignored.""" + handler = RequestHandler(dummy_manager, loop=loop) + # Bare asyncio.Transport.pause_reading() raises NotImplementedError. + handler.transport = asyncio.Transport() + + handler._pause_msg_queue_reading() + + assert handler._msg_queue_paused is True + + +def test_resume_msg_queue_reading_ignores_unsupported_transport( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """A transport without flow control raising on resume is ignored.""" + handler = RequestHandler(dummy_manager, loop=loop) + # Bare asyncio.Transport.resume_reading() raises NotImplementedError. + handler.transport = asyncio.Transport() + handler._upgraded = True # skip the reparse branch + handler._msg_queue_paused = True + + handler._resume_msg_queue_reading() + + assert handler._msg_queue_paused is False From dfdfa9d5aad5d21f91c79fb2ceeba0f8046cb6cf Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Sun, 7 Jun 2026 01:16:13 -0500 Subject: [PATCH 41/49] [PR #12830/93a2b1c3 backport][3.14] Bound pipelined request queue per connection (#12854) --- CHANGES/12830.bugfix.rst | 1 + aiohttp/_http_parser.pyx | 20 +++++- aiohttp/base_protocol.py | 25 ++++++- aiohttp/http_parser.py | 20 ++++++ aiohttp/web_protocol.py | 68 +++++++++++++++++- docs/spelling_wordlist.txt | 1 + tests/test_http_parser.py | 72 +++++++++++++++++++ tests/test_web_functional.py | 132 ++++++++++++++++++++++++++++++++++- tests/test_web_protocol.py | 93 ++++++++++++++++++++++++ 9 files changed, 425 insertions(+), 7 deletions(-) create mode 100644 CHANGES/12830.bugfix.rst diff --git a/CHANGES/12830.bugfix.rst b/CHANGES/12830.bugfix.rst new file mode 100644 index 00000000000..d44d76da404 --- /dev/null +++ b/CHANGES/12830.bugfix.rst @@ -0,0 +1 @@ +Bounded the number of parsed-but-unhandled pipelined HTTP/1 requests buffered per connection on the server; once the queue reaches an internal limit the parser stops emitting and the transport is paused, resuming as the request handler drains the queue, so a client keeping one handler busy can no longer accumulate an unbounded backlog of pipelined requests -- by :user:`bdraco`. diff --git a/aiohttp/_http_parser.pyx b/aiohttp/_http_parser.pyx index b5e5d6fc375..5ca1ccf6b0d 100644 --- a/aiohttp/_http_parser.pyx +++ b/aiohttp/_http_parser.pyx @@ -323,6 +323,8 @@ cdef class HttpParser: list _messages bint _more_data_available bint _paused + Py_ssize_t _msg_in_flight + Py_ssize_t _max_msg_queue_size bint _eof_pending object _payload unsigned long long _content_length_expected @@ -359,6 +361,7 @@ cdef class HttpParser: size_t max_field_size=8190, payload_exception=None, bint response_with_body=True, bint read_until_eof=False, bint auto_decompress=True, + Py_ssize_t max_msg_queue_size=0, ): cparser.llhttp_settings_init(self._csettings) cparser.llhttp_init(self._cparser, mode, self._csettings) @@ -373,6 +376,8 @@ cdef class HttpParser: self._buf = bytearray() self._more_data_available = False self._paused = False + self._msg_in_flight = 0 + self._max_msg_queue_size = max_msg_queue_size self._eof_pending = False self._payload = None self._payload_error = 0 @@ -556,6 +561,11 @@ cdef class HttpParser: assert self._payload is not None self._paused = True + def message_consumed(self): + # Protocol drained a queued message; free a slot for parsing. + if self._msg_in_flight > 0: + self._msg_in_flight -= 1 + def feed_eof(self): cdef bytes desc @@ -678,12 +688,12 @@ cdef class HttpRequestParser(HttpParser): size_t max_line_size=8190, size_t max_headers=128, size_t max_field_size=8190, payload_exception=None, bint response_with_body=True, bint read_until_eof=False, - bint auto_decompress=True, + bint auto_decompress=True, Py_ssize_t max_msg_queue_size=0, ): self._init(cparser.HTTP_REQUEST, protocol, loop, limit, timer, max_line_size, max_headers, max_field_size, payload_exception, response_with_body, read_until_eof, - auto_decompress) + auto_decompress, max_msg_queue_size) cdef object _on_status_complete(self): cdef int idx1, idx2 @@ -893,6 +903,12 @@ cdef int cb_on_message_complete(cparser.llhttp_t* parser) except -1: pyparser._last_error = exc return -1 else: + if pyparser._max_msg_queue_size: + pyparser._msg_in_flight += 1 + if pyparser._msg_in_flight >= pyparser._max_msg_queue_size: + # Queue full: pause llhttp between messages. feed_data() buffers + # the remainder as tail; resumes once the queue drains. + return cparser.HPE_PAUSED return 0 diff --git a/aiohttp/base_protocol.py b/aiohttp/base_protocol.py index f1f6edc3836..df3f8c089ac 100644 --- a/aiohttp/base_protocol.py +++ b/aiohttp/base_protocol.py @@ -8,6 +8,13 @@ if TYPE_CHECKING: from .http_parser import HttpParser +# Raised by transport.pause_reading()/resume_reading() when the transport +# does not support flow control; safe to ignore. +# NOTE: Catch these with a plain try/except/pass, never contextlib.suppress(): +# pause/resume run on the hot read path and suppress() is ~6x slower than +# try/except here (it builds a context manager and unpacks this tuple per call). +PAUSE_RESUME_READING_ERRORS = (AttributeError, NotImplementedError, RuntimeError) + class BaseProtocol(asyncio.Protocol): __slots__ = ( @@ -65,9 +72,15 @@ def pause_reading(self) -> None: if self.transport is not None: try: self.transport.pause_reading() - except (AttributeError, NotImplementedError, RuntimeError): + except PAUSE_RESUME_READING_ERRORS: + # Transport lacks flow control; nothing to pause. Intentionally + # ignored (see PAUSE_RESUME_READING_ERRORS; do not use suppress). pass + def _reading_paused_for_msg_queue(self) -> bool: + """Keep the transport paused for protocol-specific reasons (overridden).""" + return False + def resume_reading(self, resume_parser: bool = True) -> None: self._reading_paused = False @@ -77,10 +90,16 @@ def resume_reading(self, resume_parser: bool = True) -> None: # Reading may have been paused again in the above call if there was a lot of # compressed data still pending. - if not self._reading_paused and self.transport is not None: + if ( + not self._reading_paused + and not self._reading_paused_for_msg_queue() + and self.transport is not None + ): try: self.transport.resume_reading() - except (AttributeError, NotImplementedError, RuntimeError): + except PAUSE_RESUME_READING_ERRORS: + # Transport lacks flow control; nothing to resume. Intentionally + # ignored (see PAUSE_RESUME_READING_ERRORS; do not use suppress). pass self._reading_paused = False diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py index 5d48dedaa6c..5b46101ddaa 100644 --- a/aiohttp/http_parser.py +++ b/aiohttp/http_parser.py @@ -275,6 +275,7 @@ def __init__( response_with_body: bool = True, read_until_eof: bool = False, auto_decompress: bool = True, + max_msg_queue_size: int = 0, ) -> None: self.protocol = protocol self.loop = loop @@ -300,6 +301,9 @@ def __init__( self._headers_parser = HeadersParser( max_line_size, max_headers, max_field_size, self.lax ) + # Stop emitting messages once this many are queued unconsumed (0 = off). + self._max_msg_queue_size = max_msg_queue_size + self._msg_in_flight = 0 @abc.abstractmethod def parse_message(self, lines: list[bytes]) -> _MsgT: ... @@ -311,6 +315,11 @@ def pause_reading(self) -> None: assert self._payload_parser is not None self._payload_parser.pause_reading() + def message_consumed(self) -> None: + """Protocol drained a queued message; free a slot for parsing.""" + if self._msg_in_flight > 0: + self._msg_in_flight -= 1 + def feed_eof(self) -> _MsgT | None: if self._payload_parser is not None: self._payload_parser.feed_eof() @@ -353,6 +362,15 @@ def feed_data( # read HTTP message (request/response line + headers), \r\n\r\n # and split by lines if self._payload_parser is None and not self._upgraded: + if ( + self._max_msg_queue_size + and self._msg_in_flight >= self._max_msg_queue_size + ): + # Queue full: buffer the rest and stop. Safe pause point; + # any preceding body is consumed before the next request + # line. Resumes via feed_data(b"") when the queue drains. + self._tail = data[start_pos:] + break pos = data.find(SEP, start_pos) # consume \r\n if pos == start_pos and not self._lines: @@ -497,6 +515,8 @@ def get_content_length() -> int | None: payload = EMPTY_PAYLOAD messages.append((msg, payload)) + if self._max_msg_queue_size: + self._msg_in_flight += 1 should_close = msg.should_close else: self._tail = data[start_pos:] diff --git a/aiohttp/web_protocol.py b/aiohttp/web_protocol.py index d0719a7481d..f1b384434e0 100644 --- a/aiohttp/web_protocol.py +++ b/aiohttp/web_protocol.py @@ -16,7 +16,7 @@ from propcache import under_cached_property from .abc import AbstractAccessLogger, AbstractStreamWriter -from .base_protocol import BaseProtocol +from .base_protocol import PAUSE_RESUME_READING_ERRORS, BaseProtocol from .helpers import DEFAULT_CHUNK_SIZE, ceil_timeout from .http import ( HttpProcessingError, @@ -37,6 +37,11 @@ __all__ = ("RequestHandler", "RequestPayloadError", "PayloadAccessError") +# Max parsed-but-unhandled pipelined requests buffered per connection before +# reading is paused. Bounds memory a client can pin by keeping one handler busy +# and pipelining behind it; reading resumes as the queue drains. +MAX_MSG_QUEUE_SIZE = 32 + if TYPE_CHECKING: import ssl @@ -146,6 +151,9 @@ class RequestHandler(BaseProtocol): "_keepalive_timeout", "_lingering_time", "_messages", + "_max_msg_queue_size", + "_msg_queue_resume_size", + "_msg_queue_paused", "_message_tail", "_handler_waiter", "_waiter", @@ -186,6 +194,13 @@ def __init__( auto_decompress: bool = True, timeout_ceil_threshold: float = 5, ): + self._max_msg_queue_size = MAX_MSG_QUEUE_SIZE + # Low-water mark: resume reading once the queue drains to half the limit + # so we refill in batches instead of churning pause/resume per request. + self._msg_queue_resume_size = MAX_MSG_QUEUE_SIZE // 2 + # Set before super().__init__ so _reading_paused_for_msg_queue() is safe + # if BaseProtocol ever triggers a resume during init. + self._msg_queue_paused = False parser = HttpRequestParser( self, loop, @@ -195,6 +210,7 @@ def __init__( max_headers=max_headers, payload_exception=RequestPayloadError, auto_decompress=auto_decompress, + max_msg_queue_size=MAX_MSG_QUEUE_SIZE, ) super().__init__(loop, parser) @@ -431,6 +447,14 @@ def data_received(self, data: bytes) -> None: # don't set result twice waiter.set_result(None) + # Queue full: pause the transport (the parser already stopped + # emitting). start() resumes as it drains the queue. + if ( + not self._msg_queue_paused + and len(self._messages) >= self._max_msg_queue_size + ): + self._pause_msg_queue_reading() + self._upgraded = upgraded if upgraded and tail: self._message_tail = tail @@ -447,6 +471,36 @@ def data_received(self, data: bytes) -> None: if eof: self.close() + def _reading_paused_for_msg_queue(self) -> bool: + return self._msg_queue_paused + + def _pause_msg_queue_reading(self) -> None: + self._msg_queue_paused = True + if self.transport is not None: + try: + self.transport.pause_reading() + except PAUSE_RESUME_READING_ERRORS: + # Transport lacks flow control; nothing to pause. Intentionally + # ignored (see PAUSE_RESUME_READING_ERRORS; do not use suppress). + pass + + def _resume_msg_queue_reading(self) -> None: + if not self._upgraded: + # Reparse buffered pipelined requests while still marked paused so + # a refill past the limit does not re-pause an already-paused + # transport; only resume below once it stayed under the limit. + self.data_received(b"") + if len(self._messages) >= self._max_msg_queue_size: + return + self._msg_queue_paused = False + if not self._reading_paused and self.transport is not None: + try: + self.transport.resume_reading() + except PAUSE_RESUME_READING_ERRORS: + # Transport lacks flow control; nothing to resume. Intentionally + # ignored (see PAUSE_RESUME_READING_ERRORS; do not use suppress). + pass + def keep_alive(self, val: bool) -> None: """Set keep-alive connection mode. @@ -579,6 +633,18 @@ async def start(self) -> None: message, payload = self._messages.popleft() + # Free a parser slot; resume reading once drained to low water so + # pipelining keeps flowing while this request is handled. + # no branch: _parser is only None after connection_lost, whose path + # exits this loop, so the None case is not reachably exercisable. + if self._parser is not None: # pragma: no branch + self._parser.message_consumed() + if ( + self._msg_queue_paused + and len(self._messages) <= self._msg_queue_resume_size + ): + self._resume_msg_queue_reading() + # time is only fetched if logging is enabled as otherwise # its thrown away and never used. start = loop.time() if self._logging_enabled else None diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index 1de034c393b..a62ffed4ae9 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -250,6 +250,7 @@ peername performant pickleable ping +pipelined pipelining pluggable plugin diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index 9b7fa83d9a6..f2e0e9ae3f6 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -159,6 +159,78 @@ def test_c_parser_loaded(): assert "RawResponseMessageC" in dir(aiohttp.http_parser) +_PIPELINED_GET = b"GET / HTTP/1.1\r\nHost: a\r\n\r\n" + + +def _build_request_parser( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, + max_msg_queue_size: int, +) -> HttpRequestParser: + return request_cls( + protocol, + loop, + DEFAULT_CHUNK_SIZE, + max_line_size=8190, + max_headers=128, + max_field_size=8190, + max_msg_queue_size=max_msg_queue_size, + ) + + +def test_max_msg_queue_size_caps_emitted_messages( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, +) -> None: + parser = _build_request_parser(request_cls, protocol, loop, 4) + messages, upgraded, _tail = parser.feed_data(_PIPELINED_GET * 10) + assert len(messages) == 4 + assert not upgraded + + +def test_max_msg_queue_size_resumes_after_consume( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, +) -> None: + limit = 4 + total = 10 + parser = _build_request_parser(request_cls, protocol, loop, limit) + messages, _upgraded, _tail = parser.feed_data(_PIPELINED_GET * total) + seen = 0 + while messages: + assert len(messages) <= limit + seen += len(messages) + for _msg, _payload in messages: + parser.message_consumed() + messages, _upgraded, _tail = parser.feed_data(b"") + assert seen == total + + +def test_max_msg_queue_size_zero_is_unbounded( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, +) -> None: + parser = _build_request_parser(request_cls, protocol, loop, 0) + messages, _upgraded, _tail = parser.feed_data(_PIPELINED_GET * 50) + assert len(messages) == 50 + + +def test_message_consumed_underflow_is_ignored( + request_cls: type[HttpRequestParser], + protocol: BaseProtocol, + loop: asyncio.AbstractEventLoop, +) -> None: + parser = _build_request_parser(request_cls, protocol, loop, 4) + # No message is in flight; consuming must not underflow the counter. + parser.message_consumed() + messages, _upgraded, _tail = parser.feed_data(_PIPELINED_GET * 4) + assert len(messages) == 4 + + def test_parse_headers(parser: Any) -> None: text = b"""GET /test HTTP/1.1\r Host: a\r diff --git a/tests/test_web_functional.py b/tests/test_web_functional.py index 231a79c2757..869a53797c5 100644 --- a/tests/test_web_functional.py +++ b/tests/test_web_functional.py @@ -30,7 +30,7 @@ from aiohttp.pytest_plugin import AiohttpClient, AiohttpServer from aiohttp.streams import StreamReader from aiohttp.typedefs import Handler -from aiohttp.web_protocol import RequestHandler +from aiohttp.web_protocol import MAX_MSG_QUEUE_SIZE, RequestHandler try: import brotlicffi as brotli @@ -1715,6 +1715,136 @@ async def handler(request): await resp.release() +async def test_http1_pipelined_requests_are_count_limited( + aiohttp_server: AiohttpServer, + monkeypatch: pytest.MonkeyPatch, +) -> None: + """Requests pipelined behind a busy handler must not grow unbounded. + + A client can keep one handler active and pipeline many complete requests + behind it; the per-connection queue stays bounded by MAX_MSG_QUEUE_SIZE. + """ + pipelined_requests = 500 + slow_handler_started = asyncio.Event() + queue_observed = asyncio.Event() + max_queued = 0 + data_received = RequestHandler.data_received + + def observe_data_received(self: RequestHandler, data: bytes) -> None: + nonlocal max_queued + data_received(self, data) + if self._request_in_progress and self._messages: + max_queued = max(max_queued, len(self._messages)) + queue_observed.set() + + monkeypatch.setattr(RequestHandler, "data_received", observe_data_received) + + async def slow_handler(request: web.Request) -> web.Response: + slow_handler_started.set() + await asyncio.sleep(0.5) + return web.Response(text="slow") + + async def fast_handler(request: web.Request) -> NoReturn: + # The pipelined requests are only counted, never handled: the test + # closes the connection while the slow handler still holds the loop. + assert False + + app = web.Application() + app.router.add_get("/slow", slow_handler) + app.router.add_get("/x", fast_handler) + server = await aiohttp_server(app) + + def raw_get(path: str) -> bytes: + return ( + f"GET {path} HTTP/1.1\r\nHost: localhost\r\n" + "Connection: keep-alive\r\n\r\n" + ).encode("ascii") + + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write(raw_get("/slow")) + await writer.drain() + await asyncio.wait_for(slow_handler_started.wait(), 1) + + writer.write(raw_get("/x") * pipelined_requests) + await writer.drain() + await asyncio.wait_for(queue_observed.wait(), 1) + finally: + writer.close() + with suppress(ConnectionResetError, BrokenPipeError): + await writer.wait_closed() + + # Tight lower bound also catches over-aggressive pausing (e.g. clamping to 1). + assert MAX_MSG_QUEUE_SIZE // 2 < max_queued <= MAX_MSG_QUEUE_SIZE + + +async def test_http1_pipelined_queue_resumes_after_drain( + aiohttp_server: AiohttpServer, + monkeypatch: pytest.MonkeyPatch, +) -> None: + """A paused pipeline queue resumes reading once handlers drain it. + + Once enough requests are pipelined behind a busy handler to fill the queue, + reading is paused; as the handlers drain the queue past the low-water mark + reading must resume so the remaining buffered requests are still served. + """ + # Several times the limit so the queue refills and re-pauses while draining. + pipelined_requests = MAX_MSG_QUEUE_SIZE * 3 + first_started = asyncio.Event() + release_first = asyncio.Event() + resumed = asyncio.Event() + handled: list[str] = [] + all_handled = asyncio.Event() + + resume = RequestHandler._resume_msg_queue_reading + + def observe_resume(self: RequestHandler) -> None: + resume(self) + resumed.set() + + monkeypatch.setattr(RequestHandler, "_resume_msg_queue_reading", observe_resume) + + async def handler(request: web.Request) -> web.Response: + if request.path == "/first": + first_started.set() + await release_first.wait() + handled.append(request.path) + if len(handled) == pipelined_requests + 1: + all_handled.set() + return web.Response() + + app = web.Application() + app.router.add_get("/{tail:.*}", handler) + server = await aiohttp_server(app) + + def raw_get(path: str) -> bytes: + return ( + f"GET {path} HTTP/1.1\r\nHost: localhost\r\n" + "Connection: keep-alive\r\n\r\n" + ).encode("ascii") + + reader, writer = await asyncio.open_connection(server.host, server.port) + try: + writer.write(raw_get("/first")) + await writer.drain() + await asyncio.wait_for(first_started.wait(), 1) + + writer.write(b"".join(raw_get(f"/r{i}") for i in range(pipelined_requests))) + await writer.drain() + + # Let the busy handler finish so the queue drains and reading resumes. + release_first.set() + await asyncio.wait_for(resumed.wait(), 5) + # Every pipelined request is still served only if reading resumed. + await asyncio.wait_for(all_handled.wait(), 5) + finally: + writer.close() + with suppress(ConnectionResetError, BrokenPipeError): + await writer.wait_closed() + + assert len(handled) == pipelined_requests + 1 + + @pytest.mark.parametrize("decompressed_size", [4 * 1024 * 1024, 32 * 1024 * 1024]) async def test_unread_compressed_body_drain_is_bounded( aiohttp_server: AiohttpServer, diff --git a/tests/test_web_protocol.py b/tests/test_web_protocol.py index f823986b7b2..7e84356b1fe 100644 --- a/tests/test_web_protocol.py +++ b/tests/test_web_protocol.py @@ -48,3 +48,96 @@ def test_data_received_calls_data_received_cb( cb.assert_called_once() dummy_reader[1].feed_data.assert_called_once_with(b"x") + + +def test_pause_msg_queue_reading_without_transport( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """Pausing with no transport still records the paused state.""" + handler = RequestHandler(dummy_manager, loop=loop) + handler.transport = None + + handler._pause_msg_queue_reading() + + assert handler._msg_queue_paused is True + + +def test_resume_msg_queue_reading_after_upgrade_skips_reparse( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """Resume after an upgrade clears the pause and resumes without reparsing.""" + handler = RequestHandler(dummy_manager, loop=loop) + transport = mock.Mock() + handler.transport = transport + handler._upgraded = True + handler._msg_queue_paused = True + handler._reading_paused = False + + with mock.patch.object(RequestHandler, "data_received") as data_received: + handler._resume_msg_queue_reading() + + data_received.assert_not_called() + assert handler._msg_queue_paused is False + transport.resume_reading.assert_called_once_with() + + +def test_resume_msg_queue_reading_without_transport( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """Resume clears the pause but does not touch a missing transport.""" + handler = RequestHandler(dummy_manager, loop=loop) + handler.transport = None + handler._upgraded = True # skip the reparse branch + handler._msg_queue_paused = True + + handler._resume_msg_queue_reading() + + assert handler._msg_queue_paused is False + + +def test_resume_reading_stays_paused_for_msg_queue( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """Base resume_reading must not un-pause the transport while queue-paused.""" + handler = RequestHandler(dummy_manager, loop=loop) + transport = mock.Mock() + handler.transport = transport + handler._msg_queue_paused = True + + handler.resume_reading() + + transport.resume_reading.assert_not_called() + + +def test_pause_msg_queue_reading_ignores_unsupported_transport( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """A transport without flow control raising on pause is ignored.""" + handler = RequestHandler(dummy_manager, loop=loop) + # Bare asyncio.Transport.pause_reading() raises NotImplementedError. + handler.transport = asyncio.Transport() + + handler._pause_msg_queue_reading() + + assert handler._msg_queue_paused is True + + +def test_resume_msg_queue_reading_ignores_unsupported_transport( + loop: asyncio.AbstractEventLoop, + dummy_manager: Server, +) -> None: + """A transport without flow control raising on resume is ignored.""" + handler = RequestHandler(dummy_manager, loop=loop) + # Bare asyncio.Transport.resume_reading() raises NotImplementedError. + handler.transport = asyncio.Transport() + handler._upgraded = True # skip the reparse branch + handler._msg_queue_paused = True + + handler._resume_msg_queue_reading() + + assert handler._msg_queue_paused is False From 71f16a2a362bf7bd877e1a317f051a006abdf93d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 15:28:49 +0000 Subject: [PATCH 42/49] Bump filelock from 3.29.0 to 3.29.1 (#12806) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.29.0 to 3.29.1.
Release notes

Sourced from filelock's releases.

3.29.1

What's Changed

New Contributors

Full Changelog: https://github.com/tox-dev/filelock/compare/3.29.0...3.29.1

Changelog

Sourced from filelock's changelog.

########### Changelog ###########

3.29.1 (2026-06-03)

  • 🐛 fix(soft): refuse to follow symlinks when reading the lock file :pr:548 - by :user:dxbjavid
  • [pre-commit.ci] pre-commit autoupdate :pr:547 - by :user:pre-commit-ci[bot]
  • [pre-commit.ci] pre-commit autoupdate :pr:546 - by :user:pre-commit-ci[bot]
  • chore: improve filelock maintenance path :pr:545 - by :user:lphuc2250gma
  • chore: improve filelock maintenance path :pr:544 - by :user:lphuc2250gma
  • chore: improve filelock maintenance path :pr:542 - by :user:lphuc2250gma
  • docs: clarify per-thread scope of FileLock configuration :pr:543 - by :user:Gares95
  • [pre-commit.ci] pre-commit autoupdate :pr:541 - by :user:pre-commit-ci[bot]
  • docs: fix API docs of release() :pr:540 - by :user:MrAnno
  • [pre-commit.ci] pre-commit autoupdate :pr:539 - by :user:pre-commit-ci[bot]
  • [pre-commit.ci] pre-commit autoupdate :pr:538 - by :user:pre-commit-ci[bot]
  • [pre-commit.ci] pre-commit autoupdate :pr:537 - by :user:pre-commit-ci[bot]
  • build(deps): bump astral-sh/setup-uv from 8.0.0 to 8.1.0 :pr:536 - by :user:dependabot[bot]
  • [pre-commit.ci] pre-commit autoupdate :pr:535 - by :user:pre-commit-ci[bot]

3.29.0 (2026-04-19)

  • ✨ feat(soft): enable stale lock detection on Windows :pr:534
  • 🐛 fix(async): use single-thread executor for lock consistency :pr:533
  • build(deps): bump actions/upload-artifact from 7.0.0 to 7.0.1 :pr:530 - by :user:dependabot[bot]

3.28.0 (2026-04-14)

  • 🐛 fix(ci): unbreak release workflow, publish to PyPI again :pr:529

3.26.1 (2026-04-09)

  • 🐛 fix(asyncio): add exit to BaseAsyncFileLock and fix del loop handling :pr:518 - by :user:naarob
  • build(deps): bump pypa/gh-action-pypi-publish from 1.13.0 to 1.14.0 :pr:525 - by :user:dependabot[bot]

3.26.0 (2026-04-06)

  • ✨ feat(soft): add PID inspection and lock breaking :pr:524
  • [pre-commit.ci] pre-commit autoupdate :pr:523 - by :user:pre-commit-ci[bot]

... (truncated)

Commits
  • 438b6fe Release 3.29.1
  • bfbfa76 🐛 fix(soft): refuse to follow symlinks when reading the lock file (#548)
  • c51a72c [pre-commit.ci] pre-commit autoupdate (#547)
  • cc05fd7 [pre-commit.ci] pre-commit autoupdate (#546)
  • cb947e5 chore: improve filelock maintenance path (#545)
  • e087ca9 chore: improve filelock maintenance path (#544)
  • f9dd949 chore: improve filelock maintenance path (#542)
  • 9200f1f docs: clarify per-thread scope of FileLock configuration (#543)
  • 9d8985f [pre-commit.ci] pre-commit autoupdate (#541)
  • 7d1f48c docs: fix API docs of release() (#540)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=filelock&package-manager=pip&previous-version=3.29.0&new-version=3.29.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/lint.txt | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 4118e9578a9..c294cb16fe3 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -74,7 +74,7 @@ exceptiongroup==1.3.1 # via pytest execnet==2.1.2 # via pytest-xdist -filelock==3.29.0 +filelock==3.29.1 # via # python-discovery # virtualenv diff --git a/requirements/dev.txt b/requirements/dev.txt index 436eebebc7d..793a4b9c1ed 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -72,7 +72,7 @@ exceptiongroup==1.3.1 # via pytest execnet==2.1.2 # via pytest-xdist -filelock==3.29.0 +filelock==3.29.1 # via # python-discovery # virtualenv diff --git a/requirements/lint.txt b/requirements/lint.txt index 3c64b3a496c..9caef56aff0 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -30,7 +30,7 @@ distlib==0.4.1 # via virtualenv exceptiongroup==1.3.1 # via pytest -filelock==3.29.0 +filelock==3.29.1 # via # python-discovery # virtualenv From d797777d2191da3c190b7214456e3de6f37f287b Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 18:19:08 +0100 Subject: [PATCH 43/49] [PR #12857/69dff14d backport][3.15] Drop list compression (#12859) **This is a backport of PR #12857 as merged into master (69dff14d94b11c81bca6df3982c09a1176a1ca12).** Co-authored-by: Sam Bull --- aiohttp/streams.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aiohttp/streams.py b/aiohttp/streams.py index e1a5b531470..454153b31d3 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -566,7 +566,7 @@ def _read_nowait(self, n: int) -> bytes: count = len(self._buffer) if count == 1: return self._read_nowait_chunk(-1) - return b"".join([self._read_nowait_chunk(-1) for _ in range(count)]) + return b"".join(self._read_nowait_chunk(-1) for _ in range(count)) chunks: list[bytes] = [] while self._buffer: From 8f3100960faba4fea56afa1646aa072c128cc9db Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 18:19:23 +0100 Subject: [PATCH 44/49] [PR #12857/69dff14d backport][3.14] Drop list compression (#12858) **This is a backport of PR #12857 as merged into master (69dff14d94b11c81bca6df3982c09a1176a1ca12).** Co-authored-by: Sam Bull --- aiohttp/streams.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aiohttp/streams.py b/aiohttp/streams.py index e1a5b531470..454153b31d3 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -566,7 +566,7 @@ def _read_nowait(self, n: int) -> bytes: count = len(self._buffer) if count == 1: return self._read_nowait_chunk(-1) - return b"".join([self._read_nowait_chunk(-1) for _ in range(count)]) + return b"".join(self._read_nowait_chunk(-1) for _ in range(count)) chunks: list[bytes] = [] while self._buffer: From d9f3ec73abe5c4d1b3a3f84b43a0bd51f292481d Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Sun, 7 Jun 2026 19:22:59 +0100 Subject: [PATCH 45/49] Remove coverage ignore (#12860) --- aiohttp/web_protocol.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aiohttp/web_protocol.py b/aiohttp/web_protocol.py index 96cd9401b5d..3a5be452090 100644 --- a/aiohttp/web_protocol.py +++ b/aiohttp/web_protocol.py @@ -664,7 +664,7 @@ async def start(self) -> None: # pipelining keeps flowing while this request is handled. # no branch: _parser is only None after connection_lost, whose path # exits this loop, so the None case is not reachably exercisable. - if self._parser is not None: # pragma: no branch + if self._parser is not None: self._parser.message_consumed() if ( self._msg_queue_paused From 59684b5c270d03f7e87614a6cebeb660cd6340df Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Sun, 7 Jun 2026 19:30:59 +0100 Subject: [PATCH 46/49] Revert "Drop list compression (#12857)" (#12861) --- aiohttp/streams.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aiohttp/streams.py b/aiohttp/streams.py index 8d089fb8e1d..72d26e607d7 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -549,7 +549,7 @@ def _read_nowait(self, n: int) -> bytes: count = len(self._buffer) if count == 1: return self._read_nowait_chunk(-1) - return b"".join(self._read_nowait_chunk(-1) for _ in range(count)) + return b"".join([self._read_nowait_chunk(-1) for _ in range(count)]) chunks: list[bytes] = [] while self._buffer: From 82465ef75a536c9bd23877ae43efc3a22dfd82b1 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 19:55:35 +0100 Subject: [PATCH 47/49] [PR #12861/59684b5c backport][3.15] Revert "Drop list compression (#12857)" (#12863) **This is a backport of PR #12861 as merged into master (59684b5c270d03f7e87614a6cebeb660cd6340df).** Co-authored-by: Sam Bull --- aiohttp/streams.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aiohttp/streams.py b/aiohttp/streams.py index 454153b31d3..e1a5b531470 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -566,7 +566,7 @@ def _read_nowait(self, n: int) -> bytes: count = len(self._buffer) if count == 1: return self._read_nowait_chunk(-1) - return b"".join(self._read_nowait_chunk(-1) for _ in range(count)) + return b"".join([self._read_nowait_chunk(-1) for _ in range(count)]) chunks: list[bytes] = [] while self._buffer: From 38b956c617c8529f7e97e55e0390a474c6cb5f8a Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Sun, 7 Jun 2026 19:55:49 +0100 Subject: [PATCH 48/49] [PR #12861/59684b5c backport][3.14] Revert "Drop list compression (#12857)" (#12862) **This is a backport of PR #12861 as merged into master (59684b5c270d03f7e87614a6cebeb660cd6340df).** Co-authored-by: Sam Bull --- aiohttp/streams.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aiohttp/streams.py b/aiohttp/streams.py index 454153b31d3..e1a5b531470 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -566,7 +566,7 @@ def _read_nowait(self, n: int) -> bytes: count = len(self._buffer) if count == 1: return self._read_nowait_chunk(-1) - return b"".join(self._read_nowait_chunk(-1) for _ in range(count)) + return b"".join([self._read_nowait_chunk(-1) for _ in range(count)]) chunks: list[bytes] = [] while self._buffer: From 9c35d03aa5fecd294510196e07f176f1a2e7fa33 Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Sun, 7 Jun 2026 20:37:01 +0100 Subject: [PATCH 49/49] Release v3.14.1 (#12864) --- CHANGES.rst | 106 +++++++++++++++++++++++++++++++++++++++ CHANGES/12497.bugfix.rst | 1 - CHANGES/12795.bugfix.rst | 1 - CHANGES/12817.bugfix.rst | 1 - CHANGES/12824.bugfix.rst | 1 - CHANGES/12825.bugfix.rst | 1 - CHANGES/12826.bugfix.rst | 1 - CHANGES/12827.bugfix.rst | 1 - CHANGES/12828.bugfix.rst | 1 - CHANGES/12830.bugfix.rst | 1 - CHANGES/12831.bugfix.rst | 1 - CHANGES/12832.bugfix.rst | 1 - CHANGES/12835.bugfix.rst | 1 - aiohttp/__init__.py | 2 +- 14 files changed, 107 insertions(+), 13 deletions(-) delete mode 100644 CHANGES/12497.bugfix.rst delete mode 100644 CHANGES/12795.bugfix.rst delete mode 100644 CHANGES/12817.bugfix.rst delete mode 100644 CHANGES/12824.bugfix.rst delete mode 100644 CHANGES/12825.bugfix.rst delete mode 100644 CHANGES/12826.bugfix.rst delete mode 100644 CHANGES/12827.bugfix.rst delete mode 100644 CHANGES/12828.bugfix.rst delete mode 100644 CHANGES/12830.bugfix.rst delete mode 100644 CHANGES/12831.bugfix.rst delete mode 100644 CHANGES/12832.bugfix.rst delete mode 100644 CHANGES/12835.bugfix.rst diff --git a/CHANGES.rst b/CHANGES.rst index 3f569fd5b45..09d5c2efcaf 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -10,6 +10,112 @@ .. towncrier release notes start +3.14.1 (2026-06-07) +=================== + +Bug fixes +--------- + +- Fixed a race condition in :py:class:`~aiohttp.TCPConnector` where closing the connector while a DNS resolution was in-flight could raise :py:exc:`AttributeError` instead of :py:exc:`~aiohttp.ClientConnectionError` -- by :user:`goingforstudying-ctrl`. + + + *Related issues and pull requests on GitHub:* + :issue:`12497`. + + + +- Fixed ``CancelledError`` not closing a connection -- by :user:`aiolibsbot`. + + + *Related issues and pull requests on GitHub:* + :issue:`12795`. + + + +- Tightened up some websocket parser checks -- by :user:`Dreamsorcerer`. + + + *Related issues and pull requests on GitHub:* + :issue:`12817`. + + + +- Fixed :class:`~aiohttp.CookieJar` dropping the host-only flag of cookies when persisted with :meth:`~aiohttp.CookieJar.save` and reloaded with :meth:`~aiohttp.CookieJar.load`, so a cookie set without a ``Domain`` attribute is again scoped to the exact host that set it after a reload; the absolute expiration deadline is now persisted as well, so a reloaded cookie keeps its original lifetime instead of being rescheduled from the load time. :meth:`~aiohttp.CookieJar.load` now replaces the jar contents rather than merging onto prior state, and loaded cookies pass through the same acceptance rules as :meth:`~aiohttp.CookieJar.update_cookies`, so a cookie for an IP-address host is dropped when loaded into a jar created without ``unsafe=True`` -- by :user:`bdraco`. + + + *Related issues and pull requests on GitHub:* + :issue:`12824`. + + + +- Scoped :class:`~aiohttp.DigestAuthMiddleware` credentials to the origin of the first request it handles, so a redirect to a different origin no longer triggers a digest response computed from the configured credentials; a challenge from another origin is only answered when that origin falls within a protection space advertised by the anchor origin through the RFC 7616 ``domain`` directive -- by :user:`bdraco`. + + + *Related issues and pull requests on GitHub:* + :issue:`12825`. + + + +- Fixed the C HTTP parser not enforcing ``max_line_size`` on a request target or response reason phrase that is split across multiple reads; each fragment was checked on its own, so an accumulated line could exceed the limit without raising ``LineTooLong``. The accumulated length is now checked, matching the pure-Python parser -- by :user:`bdraco`. + + + *Related issues and pull requests on GitHub:* + :issue:`12826`. + + + +- Changed :class:`~aiohttp.TCPConnector` to reject legacy non-canonical numeric IPv4 host forms such as ``2130706433``, ``017700000001`` and ``127.1`` with :exc:`~aiohttp.InvalidUrlClientError`; only canonical dotted-quad IPv4 literals are now treated as IP address literals, while every other host is sent through the configured resolver -- by :user:`bdraco`. + + + *Related issues and pull requests on GitHub:* + :issue:`12827`. + + + +- Fixed :meth:`~aiohttp.StreamReader.readany` and :meth:`~aiohttp.StreamReader.read_nowait` joining data fed back into the buffer during the call (when draining below the low water mark resumes reading) into a single unbounded :class:`bytes`; a call now returns only the chunks that were buffered when it started, keeping the drain of an unread auto-decompressed request body bounded by the read buffer -- by :user:`bdraco`. + + + *Related issues and pull requests on GitHub:* + :issue:`12828`. + + + +- Bounded the number of parsed-but-unhandled pipelined HTTP/1 requests buffered per connection on the server; once the queue reaches an internal limit the parser stops emitting and the transport is paused, resuming as the request handler drains the queue, so a client keeping one handler busy can no longer accumulate an unbounded backlog of pipelined requests -- by :user:`bdraco`. + + + *Related issues and pull requests on GitHub:* + :issue:`12830`. + + + +- Fixed :meth:`aiohttp.web.Response.write_eof` skipping ``Payload.close()`` when the body write was interrupted by an error or cancellation, for example when a client disconnects mid-response; the payload close hook now runs in a ``finally`` so a :class:`~aiohttp.payload.Payload` body always releases its resources -- by :user:`bdraco`. + + + *Related issues and pull requests on GitHub:* + :issue:`12831`. + + + +- Fixed the pure-Python HTTP parser not enforcing ``max_line_size`` on a chunk-size line when the whole line arrived in a single read; the limit was only applied to chunk-size metadata split across reads. The complete-line case is now checked too, matching the split-line behavior -- by :user:`bdraco`. + + + *Related issues and pull requests on GitHub:* + :issue:`12832`. + + + +- Included the per-request ``server_hostname`` override in the :class:`~aiohttp.TCPConnector` connection pool key, so a pooled TLS connection is no longer reused for a request that sets ``server_hostname`` to a different value -- by :user:`bdraco`. + + + *Related issues and pull requests on GitHub:* + :issue:`12835`. + + + + +---- + + 3.14.0 (2026-06-01) =================== diff --git a/CHANGES/12497.bugfix.rst b/CHANGES/12497.bugfix.rst deleted file mode 100644 index 7fd5883ccbd..00000000000 --- a/CHANGES/12497.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed a race condition in :py:class:`~aiohttp.TCPConnector` where closing the connector while a DNS resolution was in-flight could raise :py:exc:`AttributeError` instead of :py:exc:`~aiohttp.ClientConnectionError` -- by :user:`goingforstudying-ctrl`. diff --git a/CHANGES/12795.bugfix.rst b/CHANGES/12795.bugfix.rst deleted file mode 100644 index d08ea779287..00000000000 --- a/CHANGES/12795.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed ``CancelledError`` not closing a connection -- by :user:`aiolibsbot`. diff --git a/CHANGES/12817.bugfix.rst b/CHANGES/12817.bugfix.rst deleted file mode 100644 index c8a35e309a0..00000000000 --- a/CHANGES/12817.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Tightened up some websocket parser checks -- by :user:`Dreamsorcerer`. diff --git a/CHANGES/12824.bugfix.rst b/CHANGES/12824.bugfix.rst deleted file mode 100644 index f8dbd169c31..00000000000 --- a/CHANGES/12824.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed :class:`~aiohttp.CookieJar` dropping the host-only flag of cookies when persisted with :meth:`~aiohttp.CookieJar.save` and reloaded with :meth:`~aiohttp.CookieJar.load`, so a cookie set without a ``Domain`` attribute is again scoped to the exact host that set it after a reload; the absolute expiration deadline is now persisted as well, so a reloaded cookie keeps its original lifetime instead of being rescheduled from the load time. :meth:`~aiohttp.CookieJar.load` now replaces the jar contents rather than merging onto prior state, and loaded cookies pass through the same acceptance rules as :meth:`~aiohttp.CookieJar.update_cookies`, so a cookie for an IP-address host is dropped when loaded into a jar created without ``unsafe=True`` -- by :user:`bdraco`. diff --git a/CHANGES/12825.bugfix.rst b/CHANGES/12825.bugfix.rst deleted file mode 100644 index 88d1bfe8c4c..00000000000 --- a/CHANGES/12825.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Scoped :class:`~aiohttp.DigestAuthMiddleware` credentials to the origin of the first request it handles, so a redirect to a different origin no longer triggers a digest response computed from the configured credentials; a challenge from another origin is only answered when that origin falls within a protection space advertised by the anchor origin through the RFC 7616 ``domain`` directive -- by :user:`bdraco`. diff --git a/CHANGES/12826.bugfix.rst b/CHANGES/12826.bugfix.rst deleted file mode 100644 index 7e095615d84..00000000000 --- a/CHANGES/12826.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed the C HTTP parser not enforcing ``max_line_size`` on a request target or response reason phrase that is split across multiple reads; each fragment was checked on its own, so an accumulated line could exceed the limit without raising ``LineTooLong``. The accumulated length is now checked, matching the pure-Python parser -- by :user:`bdraco`. diff --git a/CHANGES/12827.bugfix.rst b/CHANGES/12827.bugfix.rst deleted file mode 100644 index 9442867d363..00000000000 --- a/CHANGES/12827.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Changed :class:`~aiohttp.TCPConnector` to reject legacy non-canonical numeric IPv4 host forms such as ``2130706433``, ``017700000001`` and ``127.1`` with :exc:`~aiohttp.InvalidUrlClientError`; only canonical dotted-quad IPv4 literals are now treated as IP address literals, while every other host is sent through the configured resolver -- by :user:`bdraco`. diff --git a/CHANGES/12828.bugfix.rst b/CHANGES/12828.bugfix.rst deleted file mode 100644 index 9893577a587..00000000000 --- a/CHANGES/12828.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed :meth:`~aiohttp.StreamReader.readany` and :meth:`~aiohttp.StreamReader.read_nowait` joining data fed back into the buffer during the call (when draining below the low water mark resumes reading) into a single unbounded :class:`bytes`; a call now returns only the chunks that were buffered when it started, keeping the drain of an unread auto-decompressed request body bounded by the read buffer -- by :user:`bdraco`. diff --git a/CHANGES/12830.bugfix.rst b/CHANGES/12830.bugfix.rst deleted file mode 100644 index d44d76da404..00000000000 --- a/CHANGES/12830.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Bounded the number of parsed-but-unhandled pipelined HTTP/1 requests buffered per connection on the server; once the queue reaches an internal limit the parser stops emitting and the transport is paused, resuming as the request handler drains the queue, so a client keeping one handler busy can no longer accumulate an unbounded backlog of pipelined requests -- by :user:`bdraco`. diff --git a/CHANGES/12831.bugfix.rst b/CHANGES/12831.bugfix.rst deleted file mode 100644 index bf460ffccac..00000000000 --- a/CHANGES/12831.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed :meth:`aiohttp.web.Response.write_eof` skipping ``Payload.close()`` when the body write was interrupted by an error or cancellation, for example when a client disconnects mid-response; the payload close hook now runs in a ``finally`` so a :class:`~aiohttp.payload.Payload` body always releases its resources -- by :user:`bdraco`. diff --git a/CHANGES/12832.bugfix.rst b/CHANGES/12832.bugfix.rst deleted file mode 100644 index 00562fecd61..00000000000 --- a/CHANGES/12832.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed the pure-Python HTTP parser not enforcing ``max_line_size`` on a chunk-size line when the whole line arrived in a single read; the limit was only applied to chunk-size metadata split across reads. The complete-line case is now checked too, matching the split-line behavior -- by :user:`bdraco`. diff --git a/CHANGES/12835.bugfix.rst b/CHANGES/12835.bugfix.rst deleted file mode 100644 index 84a8ae00677..00000000000 --- a/CHANGES/12835.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Included the per-request ``server_hostname`` override in the :class:`~aiohttp.TCPConnector` connection pool key, so a pooled TLS connection is no longer reused for a request that sets ``server_hostname`` to a different value -- by :user:`bdraco`. diff --git a/aiohttp/__init__.py b/aiohttp/__init__.py index 5dfcd3841b9..1b35d23de57 100644 --- a/aiohttp/__init__.py +++ b/aiohttp/__init__.py @@ -1,4 +1,4 @@ -__version__ = "3.14.1.dev0" +__version__ = "3.14.1" from typing import TYPE_CHECKING