CVE intelligence and bounded remediation

CVE-2021-41125 — Scrapy is a high-level web crawling and scraping framework for Python

Medium CVSS 6.5

Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.

Severity
Medium
CVSS
6.5 (3.1)
Published
2021-10-06
CISA KEV
Not currently listed
Ecosystem
python/pypi
Weaknesses
CWE-200, CWE-522

Affected products

  • scrapy / scrapy
  • debian / debian_linux / 9.0

Showing 2 representative product identities from 3 source matches. Confirm exact affected versions with the linked vendor and NVD evidence.

Matched remediation archetype

Information disclosure and sensitive data exposure

This catalog composition supplies bounded fallback guidance. Explicitly reviewed curated workflows load with the complete record below.

Check exposure

  • Trace sensitive data through responses, errors, logs, metrics, traces, caches, exports, files, backups, and client bundles.
  • Identify affected subjects, tenants, retention windows, access controls, and downstream copies without opening unnecessary sensitive records.
  • Review metadata, timing, status, length, and existence signals as well as direct content disclosure.

Remediate safely

  • Minimize collection and output, apply field-level authorization and redaction at a centralized boundary, and return generic external errors.
  • Remove secrets and sensitive data from logs, artifacts, URLs, caches, and client-side bundles; rotate credentials that may have been exposed.
  • Update the affected component and add synthetic-data tests for response, error, observability, and export paths.

Authoritative sources

Complete CVE record and remediation plan

The detailed catalog view below loads this exact record, its source evidence, and the full seven-phase agentic change plan.