Proxy integrations
Configuration for the HTTP clients, crawlers and headless browsers teams already run.
Integrations
aiohttp proxy and httpx proxy setup for async Python
Configure an aiohttp proxy and an httpx proxy in async Python: credentials, env variables, SOCKS5, per-host routing, connection limits, timeouts and retries.
· 6 min read
Integrations
Airflow proxy configuration for data pipelines
Configure an Airflow proxy for scraping DAGs: credentials in Connections or a secrets backend, per-task proxies, HttpOperator extras, retries and pools.
· 7 min read
Integrations
AWS Lambda proxy setup: credentials, egress IPs and timeouts
Configure an AWS Lambda proxy: credential handling, stable egress IPs through a NAT gateway, NO_PROXY for AWS endpoints, timeouts and concurrency.
· 8 min read
Integrations
curl proxy cheat sheet: flags, SOCKS5, auth and debugging
A curl proxy cheat sheet: -x and proxy schemes, credentials, environment variables, NO_PROXY, HTTPS proxies, debugging 407s, timeouts, retries and timing.
· 8 min read
Integrations
Go HTTP proxy and Java HttpClient proxy setup
Set up a Go HTTP proxy with http.Transport and a Java HttpClient proxy with an Authenticator: SOCKS5, timeouts, connection reuse and the JDK Basic auth trap.
· 7 min read
Integrations
Kubernetes egress proxy setup for scraping workloads
Set up a Kubernetes egress proxy for scraping pods: credentials from a Secret, NO_PROXY for cluster traffic, stable egress IPs, NetworkPolicy and rotation.
· 10 min read
Integrations
Managing proxy credentials in Vault and AWS Secrets Manager
Store proxy credentials in Vault or AWS Secrets Manager: secret layout, log redaction, cached Python loaders, least-privilege policies and rotation.
· 8 min read
Integrations
Node.js proxy setup for fetch, undici and axios
Set up a Node.js proxy for fetch, undici and axios: ProxyAgent, EnvHttpProxyAgent, NODE_USE_ENV_PROXY, https-proxy-agent, SOCKS5, timeouts and retries.
· 7 min read
Integrations
How to set up a Playwright proxy in Node.js and Python
Set up a Playwright proxy at launch or per context, with authentication, SOCKS5 limits, bypass lists, sticky sessions and asset blocking to cut bandwidth.
· 7 min read
Integrations
How to set up a Puppeteer proxy with authentication
Set up a Puppeteer proxy with --proxy-server and page.authenticate, run one proxy per browser context, block heavy assets, and verify each exit IP.
· 6 min read
Integrations
How to set up a Python Requests proxy (auth, SOCKS5, retries)
Configure a Python Requests proxy correctly: the proxies dict, env vars and trust_env, SOCKS5 with remote DNS, timeouts, retries, and an exit-IP check.
· 7 min read
Integrations
Scrapy proxy setup: rotation, retries and 429 handling
Configure a Scrapy proxy with meta['proxy'], a small middleware for per-domain rotation and sticky sessions, the settings that matter, and sane 429 handling.
· 7 min read
Integrations
How to set up a Selenium proxy with authentication
Set up a Selenium proxy in Chrome and Firefox: IP allowlisting, a local authenticating forwarder, Firefox BiDi auth, Grid egress and exit-IP checks.
· 7 min read
Start with the evidence
Ask us to trace an address, send you the sourcing attestation, or price your current volume at our published rates. A named engineer will help with your technical and procurement review.
One business day, from a named engineer.