Adding a proxy to Selenium takes one line of code. Making it reliable across HTTPS pages, SOCKS5, authentication, rotation, headless execution, and parallel jobs takes a little more structure.

This tutorial uses Python and Selenium 4. The examples suit authorized web data collection, website monitoring, and browser testing from approved network locations.

Use proxies only where access is permitted. Follow applicable laws, website terms, robots guidance, and reasonable request rates.

What Should You Prepare Before Configuring a Selenium Proxy?

A proxy changes the network path used by the browser. It does not change where the Python script runs, and it does not automatically apply to other Python clients such as requests.

Python script → WebDriver → browser → proxy server → target website

That distinction matters because the browser may create background connections, reuse existing sockets, resolve hostnames through a different route, or retain cookies and cache between page loads. A proxy should therefore be treated as part of the browser-session configuration, not as a variable that can be changed safely halfway through a job.

Start with an isolated environment and a current Selenium package:

python -m venv .venv

# macOS or Linux
source .venv/bin/activate

# Windows PowerShell
.venv\Scripts\Activate.ps1

python -m pip install --upgrade selenium

Confirm that a direct session works before adding a proxy:

from selenium import webdriver

with webdriver.Chrome() as driver:
    driver.get("https://example.com")
    print(driver.title)

Current Selenium releases include Selenium Manager. If no driver path is supplied, Selenium Manager can discover, download, and cache a compatible driver. On a restricted corporate network, its driver download may also need approved outbound connectivity or a pre-provisioned driver.

Before writing the configuration, collect four values:

ValueExampleWhy it matters
Proxy schemehttp or socks5Tells the browser how to reach the proxy
Hostproxy.example.netIdentifies the gateway
Port8080Identifies the listening service
AuthenticationAllowlisted IP or credentialsDetermines whether another auth layer is required

Store secrets outside the source code:

export PROXY_HOST="proxy.example.net"
export PROXY_PORT="8080"
export PROXY_USERNAME="account-name"
export PROXY_PASSWORD="replace-with-a-secret"

For Windows PowerShell, use $env:PROXY_HOST="proxy.example.net" and the same pattern for the other values. Never print complete credential-bearing proxy URLs in logs, screenshots, CI artifacts, or exception reports.

How Do You Configure HTTP, HTTPS, and SOCKS5 Proxies?

For Chrome or Edge, the shortest unauthenticated setup is the Chromium --proxy-server argument. An HTTP proxy can carry HTTPS website traffic through the CONNECT method, so the endpoint below can be used when opening both HTTP and HTTPS pages.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--proxy-server=http://127.0.0.1:8080")

with webdriver.Chrome(options=options) as driver:
    driver.get("https://httpbin.org/ip")
    print(driver.find_element("tag name", "body").text)

Always include the scheme. A bare host:port value is less explicit and can behave differently when code moves between browsers or environments.

For SOCKS5, change the scheme:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--proxy-server=socks5://127.0.0.1:1080")

with webdriver.Chrome(options=options) as driver:
    driver.get("https://httpbin.org/ip")
    print(driver.find_element("tag name", "body").text)

When browser portability matters, use Selenium’s W3C proxy capability instead of a Chromium-only argument:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.proxy import Proxy, ProxyType

proxy = Proxy()
proxy.proxy_type = ProxyType.MANUAL
proxy.http_proxy = "127.0.0.1:8080"
proxy.ssl_proxy = "127.0.0.1:8080"
proxy.no_proxy = "localhost,127.0.0.1"

options = Options()
options.proxy = proxy

with webdriver.Chrome(options=options) as driver:
    driver.get("https://httpbin.org/ip")
    print(driver.find_element("tag name", "body").text)

The equivalent SOCKS5 capability uses socks_proxy and socks_version:

proxy = Proxy()
proxy.proxy_type = ProxyType.MANUAL
proxy.socks_proxy = "127.0.0.1:1080"
proxy.socks_version = 5
proxy.no_proxy = "localhost,127.0.0.1"

Use either --proxy-server or options.proxy in one configuration. Applying both creates competing settings and makes failures difficult to diagnose.

Firefox can be configured through network.proxy.* preferences:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

HOST = "127.0.0.1"
PORT = 8080

options = Options()
options.set_preference("network.proxy.type", 1)
options.set_preference("network.proxy.http", HOST)
options.set_preference("network.proxy.http_port", PORT)
options.set_preference("network.proxy.ssl", HOST)
options.set_preference("network.proxy.ssl_port", PORT)
options.set_preference("network.proxy.no_proxies_on", "localhost,127.0.0.1")

with webdriver.Firefox(options=options) as driver:
    driver.get("https://httpbin.org/ip")
    print(driver.find_element("tag name", "body").text)

For Firefox with SOCKS5, use the following preferences:

options = Options()
options.set_preference("network.proxy.type", 1)
options.set_preference("network.proxy.socks", "127.0.0.1")
options.set_preference("network.proxy.socks_port", 1080)
options.set_preference("network.proxy.socks_version", 5)
options.set_preference("network.proxy.socks_remote_dns", True)

network.proxy.socks_remote_dns asks Firefox to resolve destination hostnames through the SOCKS proxy. This is useful when the website should see name-resolution behavior associated with the proxy’s network rather than the local machine.

How Do You Handle Authenticated Proxies Safely?

Proxy authentication is the point where many short tutorials become unreliable. Embedding credentials in a Chromium argument is not a dependable cross-version method:

--proxy-server=http://username:password@host:port

A browser may reject that form, open a native authentication dialog, or behave differently in headless mode. Choose the authentication method according to the deployment environment:

MethodBest fitTrade-off
IP allowlistingServers or CI runners with fixed outbound IPsThe runner’s IP must remain stable
Local forwarding proxyLocal development, containers, CI, and mixed browsersAdds one managed process
Managed network layerLarge distributed automation systemsAdds infrastructure and platform-specific behavior
Browser extensionTightly controlled Chromium environmentsSensitive to browser versions, policy, and headless mode

IP allowlisting is usually the simplest option. After the runner’s outbound IP is approved, Selenium only needs the proxy endpoint:

options.add_argument("--proxy-server=http://gateway.example.net:8080")

No password enters the browser command line, profile, crash report, or screenshot. This approach is less suitable for laptops and ephemeral runners whose outbound addresses change frequently.

A local forwarding proxy is the more flexible alternative. It listens on a loopback address, accepts the browser’s unauthenticated connection, and supplies credentials to the upstream gateway:

Chrome → 127.0.0.1:8899 → authenticated upstream proxy → target website

The Selenium configuration stays simple:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--proxy-server=http://127.0.0.1:8899")

with webdriver.Chrome(options=options) as driver:
    driver.get("https://httpbin.org/ip")
    print(driver.find_element("tag name", "body").text)

The forwarder can run as a reviewed corporate component or a sidecar container. Bind it to 127.0.0.1, load upstream credentials from a secret store, and do not expose the listener publicly without access controls.

A generated browser extension should be a fallback rather than the default. Many older examples depend on Manifest V2 or extension permissions that no longer behave the same way in modern Chrome. Extension loading can also differ across regular Chrome, Chrome for Testing, enterprise-managed browsers, Selenium Grid, and headless mode.

If an extension is required, test the exact browser version and deployment channel. Confirm that the authentication handler is permitted, the extension loads in the selected browser mode, rejected credentials cannot create an endless challenge loop, and generated extension files never reveal secrets.

SOCKS username-and-password support also varies by browser and driver. For a portable Selenium design, an allowlisted endpoint or local forwarder is safer than assuming that the browser will accept SOCKS credentials directly.

How Do You Verify, Rotate, and Troubleshoot Proxies?

A page loading successfully does not prove that the intended proxy is active. Every new session should perform a startup check before the main task begins.

import json
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait

options = Options()
options.add_argument("--proxy-server=http://127.0.0.1:8080")

with webdriver.Chrome(options=options) as driver:
    driver.set_page_load_timeout(30)
    driver.get("https://httpbin.org/ip")

    body = WebDriverWait(driver, 10).until(
        lambda d: d.find_element("tag name", "body").text.strip()
    )
    data = json.loads(body)
    print("Observed exit address:", data["origin"])

For production work, prefer a controlled diagnostic endpoint. A public IP checker may be unavailable, rate-limited, or changed without notice. A complete startup check should confirm that the endpoint is reachable, the exit address or region is approved, an HTTPS page loads, and no authentication dialog is blocking the browser.

Rotate proxies at the browser-session boundary. Assign one endpoint, execute one bounded unit of work, and quit the driver before selecting another endpoint:

from contextlib import contextmanager
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

@contextmanager
def chrome_with_proxy(proxy_url: str):
    options = Options()
    options.add_argument(f"--proxy-server={proxy_url}")
    driver = webdriver.Chrome(options=options)
    driver.set_page_load_timeout(30)
    try:
        yield driver
    finally:
        driver.quit()

proxies = [
    "http://proxy-a.example:8000",
    "http://proxy-b.example:8000",
]

for proxy_url in proxies:
    with chrome_with_proxy(proxy_url) as driver:
        driver.get("https://example.com")
        print(proxy_url, driver.title)

Restarting costs time, but it resets connection pools, cookies, caches, service workers, and other browser state. Changing a setting under an active browser can leave old connections in place and produce mixed network identities. If one gateway rotates exit IPs behind a fixed hostname, use its documented session controls rather than assuming that every page load receives a new address.

Retry only failures that are plausibly temporary. A changed page locator or failed assertion is an application problem, not a reason to replace the proxy.

import time
from selenium import webdriver
from selenium.common.exceptions import TimeoutException, WebDriverException
from selenium.webdriver.chrome.options import Options

TRANSIENT_MARKERS = (
    "ERR_PROXY_CONNECTION_FAILED",
    "ERR_TUNNEL_CONNECTION_FAILED",
    "ERR_TIMED_OUT",
    "ERR_CONNECTION_RESET",
)

def load_once(url: str, proxy_url: str) -> str:
    options = Options()
    options.add_argument(f"--proxy-server={proxy_url}")

    with webdriver.Chrome(options=options) as driver:
        driver.set_page_load_timeout(30)
        driver.get(url)
        return driver.title

for attempt in range(1, 4):
    try:
        print(load_once("https://example.com", "http://127.0.0.1:8080"))
        break
    except (TimeoutException, WebDriverException) as exc:
        transient = any(marker in str(exc) for marker in TRANSIENT_MARKERS)
        if not transient or attempt == 3:
            raise
        time.sleep(2 ** (attempt - 1))

Use this table to narrow down common failures:

SymptomLikely causePractical check
ERR_PROXY_CONNECTION_FAILEDWrong host or port, blocked route, or offline endpointTest the endpoint from the same runner
ERR_TUNNEL_CONNECTION_FAILEDHTTPS CONNECT failed or was rejectedConfirm HTTPS support and destination rules
Authentication dialog or HTTP 407Missing or rejected authenticationCheck allowlisting, credentials, and auth method
Direct IP still appearsProxy setting was not applied or a host connected directlyInspect no_proxy rules and repeat the IP check
HTTP works but HTTPS failsTLS tunneling is unsupported or misconfiguredLoad a simple HTTPS page and inspect proxy logs
Headed mode works but headless failsExtension or auth behavior differsUse allowlisting or a local forwarder
Intermittent timeoutsProxy saturation, network latency, or a slow destinationRecord connection and page-ready timing separately

Do not disable TLS certificate checks as a first response. Certificate errors can indicate an untrusted corporate CA, TLS interception, or a misconfigured gateway. Correct the trust chain in the controlled environment instead of suppressing browser warnings globally.

How Do You Turn the Setup into Reliable Production Code?

A production design should keep proxy selection, driver creation, startup verification, task execution, and health reporting separate. One worker should own one WebDriver instance, one browser profile, and one proxy session. WebDriver should not be shared concurrently across tasks.

The following reusable factory supports Chrome, Edge, Firefox, HTTP, and SOCKS5 endpoints that do not require browser-level credentials:

from dataclasses import dataclass
from typing import Literal

from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
from selenium.webdriver.edge.options import Options as EdgeOptions
from selenium.webdriver.firefox.options import Options as FirefoxOptions

Browser = Literal["chrome", "edge", "firefox"]
Scheme = Literal["http", "https", "socks5"]

@dataclass(frozen=True)
class ProxyConfig:
    host: str
    port: int
    scheme: Scheme = "http"

    @property
    def url(self) -> str:
        return f"{self.scheme}://{self.host}:{self.port}"


def build_driver(
    browser: Browser,
    proxy: ProxyConfig,
    *,
    headless: bool = True,
):
    if browser == "chrome":
        options = ChromeOptions()
        if headless:
            options.add_argument("--headless=new")
        options.add_argument(f"--proxy-server={proxy.url}")
        driver = webdriver.Chrome(options=options)

    elif browser == "edge":
        options = EdgeOptions()
        if headless:
            options.add_argument("--headless=new")
        options.add_argument(f"--proxy-server={proxy.url}")
        driver = webdriver.Edge(options=options)

    elif browser == "firefox":
        options = FirefoxOptions()
        if headless:
            options.add_argument("-headless")

        options.set_preference("network.proxy.type", 1)
        options.set_preference("network.proxy.no_proxies_on", "localhost,127.0.0.1")

        if proxy.scheme == "socks5":
            options.set_preference("network.proxy.socks", proxy.host)
            options.set_preference("network.proxy.socks_port", proxy.port)
            options.set_preference("network.proxy.socks_version", 5)
            options.set_preference("network.proxy.socks_remote_dns", True)
        else:
            options.set_preference("network.proxy.http", proxy.host)
            options.set_preference("network.proxy.http_port", proxy.port)
            options.set_preference("network.proxy.ssl", proxy.host)
            options.set_preference("network.proxy.ssl_port", proxy.port)

        driver = webdriver.Firefox(options=options)

    else:
        raise ValueError(f"Unsupported browser: {browser}")

    driver.set_page_load_timeout(30)
    driver.set_script_timeout(30)
    return driver

Use it inside a cleanup boundary:

proxy = ProxyConfig("127.0.0.1", 8080, "http")
driver = build_driver("chrome", proxy)

try:
    driver.get("https://example.com")
    print(driver.title)
finally:
    driver.quit()

Keep authentication outside this factory. An authenticated upstream endpoint should be converted into an allowlisted endpoint or a credential-free local forwarding address before it reaches the browser.

Production logs should capture the proxy endpoint ID, startup verification result, approved exit region, navigation latency, timeout and 407 counts, browser version, driver version, and final task outcome. Redact secrets before emitting any URL:

def redact_proxy(proxy_url: str) -> str:
    if "@" not in proxy_url:
        return proxy_url
    scheme, rest = proxy_url.split("://", 1)
    host_part = rest.split("@", 1)[1]
    return f"{scheme}://***:***@{host_part}"

Use explicit waits for the page state that matters instead of adding fixed sleeps. Avoid combining implicit and explicit waits because total timeout behavior can become unpredictable. Cap retries, place unhealthy endpoints into a cooldown period, and maintain per-domain concurrency and request-rate limits even when several proxies are available.

A final selection guide is enough for most projects:

  1. No credentials: use --proxy-server for Chromium or the W3C capability for portability.
  2. Fixed runner IP: use IP allowlisting when supported.
  3. Changing runner IP: use a local forwarding proxy or managed network layer.
  4. SOCKS5: declare the scheme explicitly and verify remote DNS behavior.
  5. Frequent rotation: rotate at browser-session boundaries or use documented gateway sessions.
  6. Parallel jobs: allocate one driver, profile, and proxy session per worker.

青果网络代理IP - CTA Banner
Likes(45)
How to Choose an LLM Training Data Proxy in 2026: Performance, Bandwidth, Nodes, Cost
Rotating Proxies Global Proxies Web Scraping
2026-09-16

Choosing a proxy for LLM training data scraping shouldn't be based on IP pool size alone. This piece compares 7 providers across four dimensions — performance, bandwidth, nodes, and cost value.

How to Evaluate Proxy Quality: 7 Metrics That Really Matter
Proxies Provider Comparison Rotating Proxies Residential Proxies
2026-09-15

Proxy quality should be evaluated against real workloads rather than headline pool size. Test them under controlled conditions, segment the results by target and region, and prioritize the failures that directly affect business output.

HTTP vs SOCKS5 Proxies in 2026: Which One Do You Actually Need?
Global HTTP Proxies SOCKS5 Proxies HTTP Proxies
2026-09-14

Choose an HTTP proxy when the workload is limited to websites, web APIs, or HTTPS traffic. Choose SOCKS5 when an application uses non-HTTP protocols, requires TCP-level flexibility, or genuinely needs UDP relay.

Why Your Proxy IPs Keep Getting Blocked: 8 Common Reasons
Web Scraping Scraping Proxies Proxies Pool
2026-09-11

Proxy IPs are rarely blocked for one reason alone. Repeated IP use, sudden request spikes, broken session continuity, poor pool reputation, mismatched locations, protocol errors, route leaks, and target-side policy changes can all contribute.

发表
评论
返回
顶部