Rotating a proxy in Node.js is easy if the requirement is only “use a different endpoint for the next request.” Production code is harder. It must decide when to rotate, keep related requests on one route, stop unhealthy endpoints from re-entering immediately, and record enough context to explain failures.

The examples below use Node.js 20 or later with undici. They are intended for permitted website data collection, public-opinion monitoring, and other authorized data workflows.

What does a reliable proxy-rotation design need?

A reliable rotator needs four separate controls: selection, session continuity, failure handling, and target-level pacing. Randomly choosing a proxy for every request covers only the first one.

ControlPurposeTypical implementation
SelectionDistribute independent requestsRound-robin or least-recently-used selection
Session continuityKeep related requests on one routeSticky assignment by session key
Health handlingTemporarily remove failing endpointsFailure counter and cooldown timestamp
PacingKeep traffic within permitted limitsShared concurrency and rate limiter per target

Round-robin is a practical default because it is deterministic and easy to inspect. Random selection can repeatedly choose the same endpoint, especially when the pool is small. Weighted selection is useful only when routes have known capacity differences.

Rotation should follow the transaction boundary. A website collection task fetching unrelated pages can select a route per request. A cross-border logistics information query may need search, detail, and event-history requests to retain the same IP, cookies, and region until the transaction completes.

Install the HTTP client:

npm install undici

Store proxy URLs outside source control:

PROXY_URLS="http://user:pass@proxy-a.example:8000,http://user:pass@proxy-b.example:8000"

Do not print full proxy URLs in logs. They may contain credentials. Assign each endpoint a non-sensitive ID and log that ID instead.

How can round-robin proxy rotation be implemented in Node.js?

Create one reusable ProxyAgent per endpoint and advance an index for each independent request. Reusing dispatchers avoids rebuilding connection state on every call.

// proxy-pool.js
import { ProxyAgent, fetch } from 'undici';

function loadProxyUrls() {
  const urls = process.env.PROXY_URLS
    ?.split(',')
    .map((value) => value.trim())
    .filter(Boolean);

  if (!urls?.length) {
    throw new Error('PROXY_URLS must contain at least one proxy URL');
  }

  return urls;
}

export class ProxyPool {
  constructor(urls = loadProxyUrls()) {
    this.cursor = 0;
    this.entries = urls.map((url, index) => ({
      id: `proxy-${index + 1}`,
      dispatcher: new ProxyAgent(url),
    }));
  }

  next() {
    const entry = this.entries[this.cursor];
    this.cursor = (this.cursor + 1) % this.entries.length;
    return entry;
  }

  async close() {
    await Promise.all(
      this.entries.map((entry) => entry.dispatcher.close()),
    );
  }
}

export async function fetchWithProxy(url, options = {}, pool) {
  const proxy = pool.next();

  const response = await fetch(url, {
    ...options,
    dispatcher: proxy.dispatcher,
    signal: options.signal ?? AbortSignal.timeout(10_000),
  });

  return {
    response,
    proxyId: proxy.id,
  };
}

Use the pool from the application:

// app.js
import { ProxyPool, fetchWithProxy } from './proxy-pool.js';

const pool = new ProxyPool();

try {
  const urls = [
    'https://data.example.test/page/1',
    'https://data.example.test/page/2',
    'https://data.example.test/page/3',
  ];

  for (const url of urls) {
    const { response, proxyId } = await fetchWithProxy(
      url,
      {
        headers: {
          accept: 'application/json',
        },
      },
      pool,
    );

    if (!response.ok) {
      throw new Error(
        `${proxyId} returned HTTP ${response.status}`,
      );
    }

    const data = await response.json();

    console.log({
      proxyId,
      records: data.items?.length ?? 0,
    });
  }
} finally {
  await pool.close();
}

This version rotates before each request, applies a total timeout, reuses each dispatcher, and closes resources during shutdown. It does not yet distinguish between an endpoint failure and a target response.

Avoid rotating on every redirect or internal API call without understanding the workflow. If several requests share authentication or cookies, pass them through a sticky session instead.

How should retries, cooldowns, and sticky sessions work?

Retry only failures that may be temporary, and select another healthy route only when changing the route is safe for that transaction. A retry policy should not turn one failed request into an uncontrolled traffic spike.

A practical policy separates outcomes:

OutcomeRecommended action
Connection error or timeoutMark the route as failed and retry within a small limit
408, 429, 500, 502, 503, 504Respect Retry-After when present; otherwise use bounded backoff
Other 4xx responsesReturn the response; changing the proxy may not correct the request
Successful responseReset the route’s consecutive-failure count
Multi-step transactionKeep the same route unless the transaction is restarted cleanly

The following selector adds cooldowns and sticky assignments:

// resilient-pool.js
import { ProxyAgent } from 'undici';

export class ResilientProxyPool {
  constructor(urls, { cooldownMs = 30_000 } = {}) {
    this.cursor = 0;
    this.cooldownMs = cooldownMs;
    this.sessions = new Map();

    this.entries = urls.map((url, index) => ({
      id: `proxy-${index + 1}`,
      dispatcher: new ProxyAgent(url),
      failures: 0,
      cooldownUntil: 0,
    }));
  }

  select(sessionKey) {
    if (sessionKey && this.sessions.has(sessionKey)) {
      const index = this.sessions.get(sessionKey);
      const entry = this.entries[index];

      if (entry.cooldownUntil <= Date.now()) {
        return entry;
      }

      this.sessions.delete(sessionKey);
    }

    for (
      let checked = 0;
      checked < this.entries.length;
      checked++
    ) {
      const index = this.cursor;
      this.cursor = (this.cursor + 1) % this.entries.length;
      const entry = this.entries[index];

      if (entry.cooldownUntil <= Date.now()) {
        if (sessionKey) {
          this.sessions.set(sessionKey, index);
        }

        return entry;
      }
    }

    throw new Error(
      'No proxy endpoint is currently healthy',
    );
  }

  reportSuccess(entry) {
    entry.failures = 0;
    entry.cooldownUntil = 0;
  }

  reportFailure(entry) {
    entry.failures += 1;

    if (entry.failures >= 2) {
      entry.cooldownUntil =
        Date.now() + this.cooldownMs;
    }
  }

  releaseSession(sessionKey) {
    this.sessions.delete(sessionKey);
  }

  async close() {
    await Promise.all(
      this.entries.map((entry) =>
        entry.dispatcher.close(),
      ),
    );
  }
}

A request wrapper can now combine endpoint health with bounded retries:

// request.js
import { fetch } from 'undici';

const RETRYABLE_STATUS = new Set([
  408,
  429,
  500,
  502,
  503,
  504,
]);

const sleep = (ms) =>
  new Promise((resolve) => setTimeout(resolve, ms));

function getRetryAfterMs(response) {
  const value = response.headers.get('retry-after');

  if (!value) return null;

  const seconds = Number(value);

  if (Number.isFinite(seconds)) {
    return Math.max(0, seconds * 1_000);
  }

  const timestamp = Date.parse(value);

  if (Number.isNaN(timestamp)) return null;

  return Math.max(0, timestamp - Date.now());
}

export async function requestWithRotation(
  url,
  {
    pool,
    sessionKey,
    maxAttempts = 3,
    ...options
  },
) {
  let lastError;

  for (
    let attempt = 1;
    attempt <= maxAttempts;
    attempt++
  ) {
    const proxy = pool.select(sessionKey);
    let retryAfterMs = null;

    try {
      const response = await fetch(url, {
        ...options,
        dispatcher: proxy.dispatcher,
        signal: AbortSignal.timeout(10_000),
      });

      if (!RETRYABLE_STATUS.has(response.status)) {
        pool.reportSuccess(proxy);

        return {
          response,
          proxyId: proxy.id,
          attempt,
        };
      }

      retryAfterMs = getRetryAfterMs(response);
      await response.body?.cancel();

      pool.reportFailure(proxy);
      lastError = new Error(
        `Retryable HTTP ${response.status}`,
      );
    } catch (error) {
      pool.reportFailure(proxy);
      lastError = error;
    }

    if (sessionKey) {
      pool.releaseSession(sessionKey);
    }

    if (attempt < maxAttempts) {
      const backoffMs = Math.min(
        500 * 2 ** (attempt - 1),
        4_000,
      );

      const jitterMs =
        Math.floor(Math.random() * 250);

      await sleep(
        retryAfterMs ?? backoffMs + jitterMs,
      );
    }
  }

  throw lastError;
}

For a sticky transaction, reuse the same sessionKey across its requests and release it when the transaction finishes. If the route fails, restart the transaction with a clean cookie state when the application permits that behavior.

Do not silently move an authenticated session to another IP halfway through.

FAQ

Q: Should Node.js rotate the proxy on every request?

Only when requests are independent. Per-request rotation can fit separate public pages, but it may break authenticated sessions, pagination, or multi-step queries. Use one stable route for the complete transaction and rotate when the transaction ends. The correct boundary is determined by shared cookies, authentication, region, and application state.

Q: Is random selection better than round-robin rotation?

Not by default. Round-robin gives predictable distribution and simpler debugging. Random selection can choose the same endpoint repeatedly and may distribute a small sample unevenly. Weighted random or least-loaded selection becomes useful when endpoints have different capacity, latency, or regional availability and those differences are measured.

Q: Should an HTTP 403 response remove a proxy from the pool?

Not automatically. A 403 can reflect permissions, an expired session, a changed endpoint, or an invalid request. Compare the response body and test identical request settings before marking the route unhealthy. Connection errors and repeated target-specific failures provide stronger evidence that an endpoint should enter cooldown.

Q: How many retries should a proxy request use?

Two or three total attempts are usually a reasonable starting ceiling. The exact number depends on task cost and target limits. Use exponential backoff, add a small random delay, and respect Retry-After when available. A high retry ceiling can multiply traffic and make a temporary slowdown worse.

Q: Can the same pool be shared across Node.js workers?

The endpoint list can be shared, but selection state, cooldowns, concurrency, and rate limits must also be coordinated. Separate in-memory pools do not know what other workers are doing. For multiple processes or containers, keep health and pacing state in a shared service or data store, or partition the endpoints explicitly between workers.

青果网络代理IP - CTA Banner
Likes(53)
What Is a Residential IP? What Are Residential IPs Used For?
Residential Proxies Static IPs Rotating Proxies Global Proxies Provider Comparison Web Scraping
2026-10-27

A residential IP is a public IP tied to home broadband or residential networks. It helps verify regional content, cross-border e-commerce pages, ads, and localization — but it isn't a universal fix.

How to Use Proxy IPs for Data Collection: A Complete Guide from Requirements to Integration
Rotating Proxies Residential Proxies Backconnect Proxies HTTP Proxies Provider Comparison Web Scraping
2026-10-27

Integrating proxy IPs into data collection means confirming goal, request volume, session, and compliance first — then matching product type, finishing config, and validating before launch.

Residential IP vs ISP Datacenter Proxy: Source, Stability & Scenario Differences
Residential Proxies Datacenter IP Static IPs Provider Comparison Rotating Proxies Web Scraping
2026-10-26

Residential IPs come from home broadband networks; ISP datacenter proxies deploy servers in data centers with ISP-registered IPs. The real difference lies in network entry point, not IP labels.

How Do Dynamic IPs Work? Rotation Mechanics, Types, and Use Cases Explained
Rotating Proxies Rotating IP Residential Proxies Datacenter IP Provider Comparison Web Scraping
2026-10-26

A dynamic IP picks an exit from available resources by rule and rotates by time, request count, session state, or anomaly. Match the strategy to task continuity and frequency.

发表
评论
返回
顶部