[pull] master from aio-libs:master

CHANGES/12824.bugfix.rst

@@ -0,0 +1 @@
+Fixed :class:`~aiohttp.CookieJar` dropping the host-only flag of cookies when persisted with :meth:`~aiohttp.CookieJar.save` and reloaded with :meth:`~aiohttp.CookieJar.load`, so a cookie set without a ``Domain`` attribute is again scoped to the exact host that set it after a reload; the absolute expiration deadline is now persisted as well, so a reloaded cookie keeps its original lifetime instead of being rescheduled from the load time. :meth:`~aiohttp.CookieJar.load` now replaces the jar contents rather than merging onto prior state, and loaded cookies pass through the same acceptance rules as :meth:`~aiohttp.CookieJar.update_cookies`, so a cookie for an IP-address host is dropped when loaded into a jar created without ``unsafe=True`` -- by :user:`bdraco`.

CHANGES/12825.bugfix.rst

@@ -0,0 +1 @@
+Scoped :class:`~aiohttp.client_middleware_digest_auth.DigestAuthMiddleware` credentials to the origin of the first request it handles, so a redirect to a different origin no longer triggers a digest response computed from the configured credentials; a challenge from another origin is only answered when that origin falls within a protection space advertised by the anchor origin through the RFC 7616 ``domain`` directive -- by :user:`bdraco`.

CHANGES/12826.bugfix.rst

@@ -0,0 +1 @@
+Fixed the C HTTP parser not enforcing ``max_line_size`` on a request target or response reason phrase that is split across multiple reads; each fragment was checked on its own, so an accumulated line could exceed the limit without raising ``LineTooLong``. The accumulated length is now checked, matching the pure-Python parser -- by :user:`bdraco`.

CHANGES/12827.bugfix.rst

@@ -0,0 +1 @@
+Changed :class:`~aiohttp.TCPConnector` to reject legacy non-canonical numeric IPv4 host forms such as ``2130706433``, ``017700000001`` and ``127.1`` with :exc:`~aiohttp.InvalidUrlClientError`; only canonical dotted-quad IPv4 literals are now treated as IP address literals, while every other host is sent through the configured resolver -- by :user:`bdraco`.

CHANGES/12828.bugfix.rst

@@ -0,0 +1 @@
+Fixed :meth:`~aiohttp.StreamReader.readany` and :meth:`~aiohttp.StreamReader.read_nowait` joining data fed back into the buffer during the call (when draining below the low water mark resumes reading) into a single unbounded :class:`bytes`; a call now returns only the chunks that were buffered when it started, keeping the drain of an unread auto-decompressed request body bounded by the read buffer -- by :user:`bdraco`.

CHANGES/12829.bugfix.rst

@@ -0,0 +1 @@
+Fixed :class:`~aiohttp.ClientSession` with ``trust_env=True`` carrying a proxy's ``Proxy-Authorization`` header across a redirect; the environment proxy and its credentials are now re-resolved for each request, so a redirect that selects a different proxy no longer reuses the previous proxy's authentication -- by :user:`bdraco`.

CHANGES/12831.bugfix.rst

@@ -0,0 +1 @@
+Fixed :meth:`aiohttp.web.Response.write_eof` skipping ``Payload.close()`` when the body write was interrupted by an error or cancellation, for example when a client disconnects mid-response; the payload close hook now runs in a ``finally`` so a :class:`~aiohttp.payload.Payload` body always releases its resources -- by :user:`bdraco`.

CHANGES/12832.bugfix.rst

@@ -0,0 +1 @@
+Fixed the pure-Python HTTP parser not enforcing ``max_line_size`` on a chunk-size line when the whole line arrived in a single read; the limit was only applied to chunk-size metadata split across reads. The complete-line case is now checked too, matching the split-line behavior -- by :user:`bdraco`.

CHANGES/12835.bugfix.rst

@@ -0,0 +1 @@
+Included the per-request ``server_hostname`` override in the :class:`~aiohttp.TCPConnector` connection pool key, so a pooled TLS connection is no longer reused for a request that sets ``server_hostname`` to a different value -- by :user:`bdraco`.

aiohttp/_http_parser.pyx

@@ -781,7 +781,7 @@ cdef int cb_on_url(cparser.llhttp_t* parser,
                    const char *at, size_t length) except -1:
     cdef HttpParser pyparser = <HttpParser>parser.data
     try:
-        if length > pyparser._max_line_size:
+        if len(pyparser._buf) + length > pyparser._max_line_size:
             status = pyparser._buf + at[:length]
             raise LineTooLong(status[:100] + b"...", pyparser._max_line_size)
         extend(pyparser._buf, at, length)
@@ -796,7 +796,7 @@ cdef int cb_on_status(cparser.llhttp_t* parser,
                       const char *at, size_t length) except -1:
     cdef HttpParser pyparser = <HttpParser>parser.data
     try:
-        if length > pyparser._max_line_size:
+        if len(pyparser._buf) + length > pyparser._max_line_size:
             reason = pyparser._buf + at[:length]
             raise LineTooLong(reason[:100] + b"...", pyparser._max_line_size)
         extend(pyparser._buf, at, length)

aiohttp/client.py

@@ -549,10 +549,11 @@ async def _request(
         if proxy is None:
             proxy = self._default_proxy
 
+        resolved_proxy_headers: CIMultiDict[str] | None
         if proxy is None:
-            proxy_headers = None
+            resolved_proxy_headers = None
         else:
-            proxy_headers = self._prepare_headers(proxy_headers)
+            resolved_proxy_headers = self._prepare_headers(proxy_headers)
             try:
                 proxy = URL(proxy)
             except ValueError as e:
@@ -654,16 +655,17 @@ async def _request(
                     if proxy is not None:
                         proxy_ = URL(proxy)
                     elif self._trust_env:
+                        # Re-resolve per iteration; drop stale env-proxy auth so
+                        # a redirect that switches proxies can't leak credentials.
+                        resolved_proxy_headers = None
                         with suppress(LookupError):
                             proxy_, env_proxy_auth = await asyncio.to_thread(
                                 get_env_proxy_for_url, url
                             )
-                            if env_proxy_auth is not None and (
-                                proxy_headers is None
-                                or hdrs.PROXY_AUTHORIZATION not in proxy_headers
-                            ):
-                                proxy_headers = proxy_headers or CIMultiDict()
-                                proxy_headers[hdrs.PROXY_AUTHORIZATION] = env_proxy_auth
+                            if env_proxy_auth is not None:
+                                resolved_proxy_headers = CIMultiDict(
+                                    {hdrs.PROXY_AUTHORIZATION: env_proxy_auth}
+                                )
 
                     req = self._request_class(
                         method,
@@ -684,7 +686,7 @@ async def _request(
                         session=self,
                         ssl=ssl,
                         server_hostname=server_hostname,
-                        proxy_headers=proxy_headers,
+                        proxy_headers=resolved_proxy_headers,
                         traces=traces,
                         trust_env=self.trust_env,
                     )

aiohttp/client_middleware_digest_auth.py

@@ -162,6 +162,15 @@ class DigestAuthMiddleware:
     - Includes replay attack protection with client nonce count tracking
     - Supports preemptive authentication per RFC 7616 Section 3.6
 
+    Origin scoping:
+    The credentials are scoped to the origin of the first request the
+    middleware handles. A request to a different origin is passed through
+    untouched, so it never receives a digest response computed from those
+    credentials, unless that origin falls within a protection space the
+    anchor origin advertised through the RFC 7616 ``domain`` directive. Make
+    the first request through the middleware against the intended origin, as
+    the anchor is pinned to it and not reset for the life of the instance.
+
     Standards compliance:
     - RFC 7616: HTTP Digest Access Authentication (primary reference)
     - RFC 2617: HTTP Authentication (deprecated by RFC 7616)
@@ -198,6 +207,8 @@ def __init__(
         self._preemptive: bool = preemptive
         # Set of URLs defining the protection space
         self._protection_space: list[str] = []
+        # Origin the credentials are scoped to; set on the first request.
+        self._origin: URL | None = None
 
     async def _encode(self, method: str, url: URL, body: Payload | Literal[b""]) -> str:
         """
@@ -447,6 +458,16 @@ async def __call__(
         self, request: ClientRequest, handler: ClientHandlerType
     ) -> ClientResponse:
         """Run the digest auth middleware."""
+        # Credentials are scoped to the first request's origin. Other origins
+        # pass through untouched unless a challenge from the anchor origin
+        # advertised them via RFC 7616 domain; mirrors aiohttp stripping
+        # Authorization on cross-origin redirects.
+        origin = request.url.origin()
+        if self._origin is None:
+            self._origin = origin
+        elif origin != self._origin and not self._in_protection_space(request.url):
+            return await handler(request)
+
         response = None
         for retry_count in range(2):
             # Apply authorization header if:

aiohttp/client_reqrep.py

@@ -180,6 +180,7 @@ class ConnectionKey(NamedTuple):
     ssl: SSLContext | bool | Fingerprint
     proxy: URL | None
     proxy_headers_hash: int | None  # hash(CIMultiDict)
+    server_hostname: str | None = None
 
 
 class ClientResponse(HeadersMixin):
@@ -818,6 +819,7 @@ def connection_key(self) -> ConnectionKey:
                 self._ssl,
                 None,
                 None,
+                self.server_hostname,
             ),
         )
 
@@ -1055,6 +1057,7 @@ def connection_key(self) -> ConnectionKey:
                 self._ssl,
                 self.proxy,
                 h,
+                self.server_hostname,
             ),
         )
 

aiohttp/connector.py

@@ -28,6 +28,7 @@
     ClientConnectorSSLError,
     ClientHttpProxyError,
     ClientProxyConnectionError,
+    InvalidUrlClientError,
     ServerFingerprintMismatch,
     UnixClientConnectorError,
     cert_errors,
@@ -43,6 +44,7 @@
 from .helpers import (
     _SENTINEL,
     ceil_timeout,
+    is_canonical_ipv4_address,
     is_ip_address,
     sentinel,
     set_exception,
@@ -1046,6 +1048,11 @@ async def _resolve_host(
     ) -> list[ResolveResult]:
         """Resolve host and return list of addresses."""
         if is_ip_address(host):
+            # Reject legacy numeric IPv4 forms (e.g. 2130706433, 127.1) that
+            # socket would map onto an address, slipping past a connector-level
+            # policy that only sees the raw host.
+            if ":" not in host and not is_canonical_ipv4_address(host):
+                raise InvalidUrlClientError(host, "is not a canonical IPv4 address")
             return [
                 {
                     "hostname": host,

aiohttp/cookiejar.py

@@ -37,6 +37,9 @@
 _MIN_SCHEDULED_COOKIE_EXPIRATION = 100
 _SIMPLE_COOKIE = SimpleCookie()
 
+# Not persisted; the absolute deadline is saved instead.
+_RELATIVE_EXPIRY_ATTRS = frozenset(("max-age", "expires"))
+
 
 class CookieJar(AbstractCookieJar):
     """Implements cookie storage adhering to RFC 6265."""
@@ -133,21 +136,28 @@ def save(self, file_path: PathLike) -> None:
             :class:`str` or :class:`pathlib.Path` instance.
         """
         file_path = pathlib.Path(file_path)
-        data: dict[str, dict[str, dict[str, str | bool]]] = {}
+        data: dict[str, dict[str, dict[str, str | bool | float]]] = {}
         for (domain, path), cookie in self._cookies.items():
             key = f"{domain}|{path}"
             data[key] = {}
             for name, morsel in cookie.items():
-                morsel_data: dict[str, str | bool] = {
+                morsel_data: dict[str, str | bool | float] = {
                     "key": morsel.key,
                     "value": morsel.value,
                     "coded_value": morsel.coded_value,
                 }
-                # Save all morsel attributes that have values
+                # Skip relative expiry; the absolute deadline is saved below.
                 for attr in morsel._reserved:  # type: ignore[attr-defined]
+                    if attr in _RELATIVE_EXPIRY_ATTRS:
+                        continue
                     attr_val = morsel[attr]
                     if attr_val:
                         morsel_data[attr] = attr_val
+                # Persist or it reloads as a domain cookie and leaks to subdomains.
+                if (domain, name) in self._host_only_cookies:
+                    morsel_data["host_only"] = True
+                if (exp := self._expirations.get((domain, path, name))) is not None:
+                    morsel_data["expires_timestamp"] = exp
                 data[key][name] = morsel_data
 
         # Cookie persistence may include authentication/session tokens.
@@ -164,34 +174,33 @@ def save(self, file_path: PathLike) -> None:
     def load(self, file_path: PathLike) -> None:
         """Load cookies from a JSON file.
 
+        Replaces the current jar contents; loaded cookies pass through the
+        same acceptance rules as :meth:`update_cookies`.
+
         :param file_path: Path to file from where cookies will be
             imported, :class:`str` or :class:`pathlib.Path` instance.
         """
         file_path = pathlib.Path(file_path)
         with file_path.open(mode="r", encoding="utf-8") as f:
             data = json.load(f)
-        self._cookies = self._load_json_data(data)
+        self._load_json_data(data)
 
     def _load_json_data(
-        self, data: dict[str, dict[str, dict[str, str | bool]]]
-    ) -> defaultdict[tuple[str, str], SimpleCookie]:
-        """Load cookies from parsed JSON data."""
-        cookies: defaultdict[tuple[str, str], SimpleCookie] = defaultdict(SimpleCookie)
+        self, data: dict[str, dict[str, dict[str, str | bool | float]]]
+    ) -> None:
+        """Replace contents, routing cookies through update_cookies()."""
+        self.clear()
         for compound_key, cookie_data in data.items():
             domain, path = compound_key.split("|", 1)
-            key = (domain, path)
             for name, morsel_data in cookie_data.items():
                 morsel: Morsel[str] = Morsel()
-                morsel_key = morsel_data["key"]
-                morsel_value = morsel_data["value"]
-                morsel_coded_value = morsel_data["coded_value"]
                 # Use __setstate__ to bypass validation, same pattern
                 # used in _build_morsel and _cookie_helpers.
                 morsel.__setstate__(  # type: ignore[attr-defined]
                     {
-                        "key": morsel_key,
-                        "value": morsel_value,
-                        "coded_value": morsel_coded_value,
+                        "key": morsel_data["key"],
+                        "value": morsel_data["value"],
+                        "coded_value": morsel_data["coded_value"],
                     }
                 )
                 # Restore morsel attributes
@@ -202,8 +211,17 @@ def _load_json_data(
                         "coded_value",
                     ):
                         morsel[attr] = morsel_data[attr]
-                cookies[key][name] = morsel
-        return cookies
+                # Drop the domain so update_cookies() re-marks it host-only.
+                if morsel_data.get("host_only"):
+                    morsel["domain"] = ""
+                response_url = (
+                    URL.build(scheme="https", host=domain) if domain else URL()
+                )
+                self.update_cookies({name: morsel}, response_url)
+                # Restore the absolute deadline; update_cookies() schedules none.
+                if (exp := morsel_data.get("expires_timestamp")) is not None:
+                    self._expire_cookie(float(exp), domain, path, name)
+        self._do_expiration()
 
     def clear(self, predicate: ClearCookiePredicate | None = None) -> None:
         if predicate is None:

aiohttp/helpers.py

@@ -485,6 +485,28 @@ def is_ip_address(host: str | None) -> bool:
     return ":" in host or host.replace(".", "").isdigit()
 
 
+def is_canonical_ipv4_address(host: str) -> bool:
+    """Check if host is a canonical dotted-quad IPv4 address.
+
+    Rejects the legacy numeric forms that ``socket`` still accepts and
+    maps onto an address, e.g. ``2130706433``, ``017700000001``, ``127.1``.
+    """
+    parts = host.split(".")
+    if len(parts) != 4:
+        return False
+    for part in parts:
+        # Each octet must be 1-3 ASCII digits; reject unicode digits
+        # (which ``str.isdigit`` accepts but ``int`` may not), octal
+        # leading zeros, and values above 255.
+        if not (1 <= len(part) <= 3) or not part.isascii() or not part.isdigit():
+            return False
+        if part[0] == "0" and len(part) != 1:
+            return False
+        if int(part) > 255:
+            return False
+    return True
+
+
 _cached_current_datetime: int | None = None
 _cached_formatted_datetime = ""
 

aiohttp/http_parser.py

@@ -937,6 +937,10 @@ def feed_data(
                 if self._chunk == ChunkState.PARSE_CHUNKED_SIZE:
                     pos = chunk.find(SEP)
                     if pos >= 0:
+                        # Only chunk-size lines reach here; trailers enforce
+                        # _max_field_size separately in PARSE_TRAILERS below.
+                        if pos > self._max_line_size:
+                            raise LineTooLong(chunk[:100] + b"...", self._max_line_size)
                         i = chunk.find(CHUNK_EXT, 0, pos)
                         if i >= 0:
                             size_b = chunk[:i]  # strip chunk-extensions

aiohttp/streams.py

@@ -543,14 +543,21 @@ def _read_nowait(self, n: int) -> bytes:
         """Read not more than n bytes, or whole buffer if n == -1"""
         self._timer.assert_timeout()
 
-        chunks = []
+        if n == -1:
+            # Drain only chunks present now; _read_nowait_chunk() can
+            # re-entrantly resume_reading() and refill the buffer.
+            count = len(self._buffer)
+            if count == 1:
+                return self._read_nowait_chunk(-1)
+            return b"".join([self._read_nowait_chunk(-1) for _ in range(count)])
+
+        chunks: list[bytes] = []
         while self._buffer:
             chunk = self._read_nowait_chunk(n)
             chunks.append(chunk)
-            if n != -1:
-                n -= len(chunk)
-                if n == 0:
-                    break
+            n -= len(chunk)
+            if n == 0:
+                break
 
         return b"".join(chunks) if chunks else b""
 

aiohttp/web_response.py

@@ -688,8 +688,10 @@ async def write_eof(self, data: bytes = b"") -> None:
         if body is None or self._must_be_empty_body:
             await super().write_eof()
         elif isinstance(self._body, Payload):
-            await self._body.write(self._payload_writer)
-            await self._body.close()
+            try:
+                await self._body.write(self._payload_writer)
+            finally:
+                await self._body.close()
             await super().write_eof()
         else:
             await super().write_eof(cast(bytes, body))