Proxy 429 and 403 errors: causes and fixes

A proxy 429 or 403 is almost always the target website answering, not the proxy. A 429 Too Many Requests means you exceeded the target's rate limit; a 403 Forbidden means it refused the request, often because of the address, headers or client fingerprint it saw. The fix for a 429 is to slow down and honor Retry-After, not to retry faster from more addresses; the fix for a 403 is to find out what the target objected to. Errors that genuinely come from the proxy look different: 407 for authentication, 502, 503 or 504 from the gateway, and connection resets.

This guide shows how to tell the two sources apart, a retry policy that works with target rate limits rather than against them, and when a change of proxy type is the right answer.

Is the error from the proxy or from the target?

When a proxy 429 or 403 appears in your logs, the first question is which hop produced it. Every request through a proxy has two hops: your client to the gateway, and the gateway to the target. Either can fail, and each failure means something different.

For HTTPS targets, which is most of them, the distinction is clean. Your client first sends a CONNECT request to the gateway to open a tunnel. If the gateway cannot authenticate you or cannot reach the target, it answers the CONNECT with an error, and the tunnel never opens. Once the tunnel is open, every response you receive, including every 403 and 429, comes from the target through the encrypted tunnel. The gateway cannot read or alter it.

In Python Requests, a failed CONNECT surfaces as a requests.exceptions.ProxyError exception, not as a response object. A response object with a status code, for an HTTPS URL, is from the target. In curl -v output, the proxy's reply to CONNECT appears before the TLS handshake, and the target's status appears after it.

For plain HTTP targets, the proxy forwards the request and returns the target's response, but it can also return its own error responses. There, use timing and consistency: a gateway error usually repeats across unrelated target hosts, while a target error is specific to one site.

Errors that come from the proxy side

• 407 Proxy Authentication Required: the gateway did not accept your credentials or your source IP. This is a configuration error and will not fix itself on retry. See IP allowlist vs username and password proxy authentication for the usual causes.
• 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout on CONNECT: the gateway could not complete the connection to the target through the chosen exit, for example because the exit peer went offline or the target did not respond in time. These are usually transient and safe to retry with backoff.
• Connection resets and timeouts to the gateway itself: network problems between you and the gateway, a firewall on your side, or a wrong port. If every request fails this way, check the host, port and scheme against the dashboard.

Errors that come from the target side

• 429 Too Many Requests: you exceeded a rate limit. RFC 6585 defines the status and allows the server to include a Retry-After header saying how long to wait.
• 403 Forbidden: the server understood the request and refused it. Causes range from a blocked address range to a missing header to a deliberate access policy.
• 503 with Retry-After from the target: some sites use 503 rather than 429 for load shedding or rate limiting.
• Challenge or block pages returned as 200: many bot-management systems return a normal 200 with a challenge page, a login wall or an empty result set. Your status-code handling will not see these at all, which is why content validation matters.

Status code reference: likely cause and fix

A retry policy that respects rate limits

Retries are necessary, because transient errors are a normal part of any network path, and especially of peer-based networks where exits come and go. Retries done badly are also the most common way a scraper turns an occasional 429 into a sustained block. A sound policy has five parts.

Exponential backoff with jitter

Wait longer after each consecutive failure, and randomize the wait. Exponential growth ensures a struggling target gets relief; randomization ensures that many workers that failed at the same moment do not all retry at the same moment. A common form is "full jitter": wait a random time between zero and base × 2^attempt, capped at a maximum.

Honor Retry-After

When a 429 or 503 carries a Retry-After header, the server has told you exactly how long to wait. RFC 9110 defines it as either a number of seconds or an HTTP date. Use it in place of your computed backoff. If the requested wait is longer than your job can tolerate, stop and reschedule the work rather than retrying early.

Per-host concurrency limits

Rate limits apply per target, so your limits should too. Cap the number of in-flight requests to each host independently of the total size of your worker pool. Twenty workers spread across twenty sites and twenty workers pointed at one site are very different loads, and a global concurrency setting cannot tell them apart.

Circuit breakers

When a host returns a high share of 429, 403 or challenge responses over a short window, stop sending to it entirely for a cooling-off period, then let a small number of probe requests through before resuming. A circuit breaker turns "keep failing faster" into "pause and recover", and it keeps a blocked target from consuming bandwidth you are paying for.

Bounded attempts and idempotency

Cap the number of attempts, then fail the task and record why. Retry automatically only requests that are safe to repeat: GET, HEAD, and operations you have designed to be idempotent.

Python example: backoff that honors Retry-After

The function below implements the first, second, third and fifth parts using Python Requests. It reads the proxy URL from the PROXY_URL environment variable, in the shape http://USERNAME:[email protected]:PORT, limits concurrency per host, retries connection failures and retryable statuses with full-jitter backoff, uses Retry-After when present, and hands back the response when the server asks for a longer wait than the job should absorb.

import os
import random
import threading
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from urllib.parse import urlsplit

import requests

PROXY_URL = os.environ["PROXY_URL"]
PROXIES = {"http": PROXY_URL, "https": PROXY_URL}

RETRY_STATUSES = {429, 502, 503, 504}
MAX_ATTEMPTS = 5
BASE_DELAY = 1.0
MAX_BACKOFF = 60.0
MAX_RETRY_AFTER = 300.0
PER_HOST_CONCURRENCY = 4

_slots_lock = threading.Lock()
_host_slots: dict[str, threading.BoundedSemaphore] = {}


def host_slot(url):
    host = urlsplit(url).hostname
    with _slots_lock:
        if host not in _host_slots:
            _host_slots[host] = threading.BoundedSemaphore(PER_HOST_CONCURRENCY)
        return _host_slots[host]


def retry_after_seconds(response):
    value = response.headers.get("Retry-After", "").strip()
    if not value:
        return None
    if value.isdigit():
        return float(value)
    try:
        when = parsedate_to_datetime(value)
    except (TypeError, ValueError):
        return None
    if when.tzinfo is None:
        when = when.replace(tzinfo=timezone.utc)
    return max(0.0, (when - datetime.now(timezone.utc)).total_seconds())


def backoff_seconds(attempt):
    return random.uniform(0, min(MAX_BACKOFF, BASE_DELAY * 2**attempt))


def fetch(session, url):
    for attempt in range(MAX_ATTEMPTS):
        last_attempt = attempt == MAX_ATTEMPTS - 1
        with host_slot(url):
            try:
                response = session.get(url, proxies=PROXIES, timeout=30)
            except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
                if last_attempt:
                    raise
                response = None

        delay = None
        if response is not None:
            if response.status_code == 407:
                raise RuntimeError("proxy rejected credentials; check PROXY_URL")
            if response.status_code not in RETRY_STATUSES or last_attempt:
                return response
            delay = retry_after_seconds(response)
            if delay is not None and delay > MAX_RETRY_AFTER:
                return response

        time.sleep(delay if delay is not None else backoff_seconds(attempt))

A few notes on the choices. The concurrency slot is released before sleeping, so a waiting retry does not block other requests to the same host from making progress. A 407 raises immediately, because retrying bad credentials only produces more 407s. A 403 is returned to the caller without retrying, because repeating a refused request unchanged rarely changes the answer; the caller should decide whether to adjust the request, the session or the proxy type. For the Requests-specific proxy configuration behind this, including SOCKS5 and session reuse, see using a proxy with Python Requests.

Rotate or slow down?

Rotating to a new exit address is the right response to some failures and the wrong response to others.

Slow down when the target returns 429, when it returns 503 with Retry-After, or when challenge pages increase as your rate increases. These are rate signals. The target is telling you the load is too high, and more addresses carrying the same load does not change that. It also runs against the site's stated limits, which your use should respect.

Rotate when a single exit returns errors that others do not, such as a gateway 502 because a peer dropped, or a 403 tied to that one address's reputation while other addresses succeed at the same rate. For flows that need one address throughout, such as logins and carts, rotation means restarting the flow on a new sticky session rather than switching mid-flow; rotating vs sticky proxies covers how to structure that.

Stop when the target's terms or robots.txt exclude the content, when a login wall indicates the data is not public, or when the target is clearly signaling that automated access is unwelcome. Proxies do not change what a site permits. Every ProxyForge account is bound by the acceptable use policy, which asks you to respect published rate limits, back off on error responses and not spread traffic across addresses to get around a limit a destination has set. Circumventing authentication or access controls to obtain data you are not entitled to is prohibited outright.

Header and TLS consistency

Many 403s and challenge pages are not about the IP at all. Bot-management systems compare what a client claims to be with how it behaves. The common inconsistencies:

• A User-Agent claiming to be a current desktop browser, sent by an HTTP library whose TLS handshake looks nothing like a browser's.
• Missing headers that a real browser always sends, such as Accept-Language, or headers in an order no browser uses.
• An Accept-Language or time zone that contradicts the exit country.
• Cookies set on one address and replayed from another, or a single cookie jar shared across many rotating addresses.

The fix is consistency, not disguise. Send an honest, stable User-Agent that identifies your client where the target's policy expects it, keep headers coherent with the exit country, and scope cookies to one session and one address. If a target requires a full browser to serve its content, use a real browser through the proxy rather than an HTTP library imitating one.

When to move to a different proxy type

If errors persist after you have fixed rate, concurrency and consistency, the address type may be the mismatch:

• Datacenter to ISP or residential: some targets score datacenter ranges more strictly than consumer ranges. If a target returns 403 for datacenter addresses at modest rates, and the same request succeeds from a consumer connection, a consumer-registered address may be appropriate. ISP proxies are static and dedicated; residential are rotating or sticky.
• Residential to ISP: if failures cluster around sticky sessions ending early, a static ISP address removes that variable for long flows.
• Residential to mobile: some mobile-first services treat carrier addresses differently from fixed-line ones.

Change one variable at a time and measure. Switching proxy type while also changing rate and headers leaves you unable to say what worked.

Next steps

Most proxy 429 problems are solved on the client side: slower per-host rates, backoff that honors Retry-After, circuit breakers, and consistent sessions. Most 403 problems are diagnosed by reading the response body and testing one variable at a time. The proxy's job is to give you clean, clearly sourced exits and clear errors when its own hop fails.

ProxyForge supplies residential, mobile, ISP and datacenter addresses over HTTP, HTTPS and SOCKS5, with consent-based sourcing documented on the sourcing page. If you are diagnosing persistent errors on a specific target, a named engineer can review your setup with you, and the contact page is the fastest route.