When the number of monitored targets grows from dozens to thousands, the most intuitive response is to add threads, raise concurrency, and purchase a larger proxy plan. The result is often amusing from an engineering perspective: the program becomes much busier, the bill becomes much larger, but the volume of valid data does not grow proportionally.
The real challenge is not choosing a single “maximum concurrency” value. It is managing three interdependent budgets: the concurrency budget determines instantaneous load, the bandwidth budget determines data transfer overhead, and the cost per valid result determines whether the workload is worth running at all.
Why Do Costs Suddenly Get Out of Control as Task Volume Grows?
Because task volume is only the visible variable. Retry amplification, oversized responses, and scheduling collisions are usually the main cost drivers.
Suppose a monitoring workload originally visits 100,000 pages per day, with an average response size of 200 KB. Under normal conditions, the total transferred data would be approximately 20 GB. If the success rate drops after scaling and each task is retried 1.5 times on average, the system will issue about 250,000 requests and transfer approximately 50 GB.
This estimate does not yet include redirect pages, images, scripts, or failed compression. If the client mistakenly downloads an entire page when the business only needs a title, price, or status field, an ordinary task can easily transfer ten times more data than necessary.
Before scaling, break the workload down into four multipliers:
Total requests = Number of tasks × Fetch frequency × Average number of attempts
Total traffic = Total requests × Average response size in bytes
Valid results = Total requests × Business success rate × Deduplication retention rate
Cost per valid result = Proxy cost ÷ Number of valid resultsLooking only at total request volume misses two expensive problems: how many attempts each task actually makes and how much usable data remains after processing.
How Should the Concurrency Budget Be Allocated?
Do not configure one global concurrency value for the entire application. Apply separate limits by target, task priority, and proxy endpoint.
Monitoring systems usually access multiple targets at the same time. These targets differ in response speed, acceptable request frequency, and business value. Putting all of them into a single thread pool allows slow targets to occupy all available connections and low-value jobs to displace critical monitoring tasks.
The following table can be used as a starting point for load testing:
| Task type | Initial concurrency per target | Initial timeout | Retry count | Scheduling method |
|---|---|---|---|---|
| High-frequency status monitoring | 2–4 | 5–8 seconds | 1 | Fixed intervals with random jitter |
| Standard listing updates | 1–3 | 8–12 seconds | 1 | Batched polling |
| Low-frequency detail enrichment | 1–2 | 10–15 seconds | 0–1 | Run in a low-priority queue during off-peak periods |
| Historical data backfill | 1 | 15–20 seconds | 0 | Separate rate-limited queue |
These values are only starting points for testing, not universal settings. During operation, focus on requests per second, P95 response time, timeout rate, and valid-result rate. If P95 continues to increase, reduce concurrency first. If responses remain stable while the queue keeps growing, increase concurrency in small increments.
A more reliable scheduler should have at least two layers of limits:
global_limit = Semaphore(20)
host_limits = {
"target_a": Semaphore(3),
"target_b": Semaphore(2),
}
async def run_task(task):
async with global_limit:
async with host_limits[task.host]:
result = await fetch(task, timeout=8)
if result.retryable and task.retries < 1:
await sleep(random.uniform(1.5, 3.0))
return await run_task(task.next_retry())
return resultThe global limit protects the proxy endpoint and local resources, while the per-target limit controls the request rate for each destination. Adding a randomized delay before a retry prevents a batch of failed tasks from restarting in the same second and creating a retry storm.
How Can Retry Logic Be Prevented from Driving Up Proxy Consumption?
Retries must be constrained by error type, retry count, and time window. Not every failed request should be sent again.
Connection timeouts, proxy connection failures, and temporary server errors may justify a retry. Invalid parameters, insufficient permissions, deleted pages, and business parsing failures usually should not be handled by repeatedly sending requests through the proxy layer.
| Failure type | Retry? | Recommended action |
|---|---|---|
| Connection establishment timeout | Yes | Retry once after a delay |
| Proxy node connection failure | Yes | Change the connection and retry once |
| Temporary server error | With caution | Apply a target-specific backoff window |
| Invalid request parameters | No | Correct the task configuration |
| Page structure change | No | Send the task to the parsing exception queue |
| Duplicate response content | No | Inspect scheduling and deduplication logic |
One of the most common engineering mistakes is implementing separate retry logic in the network layer, business layer, and task queue. Each layer may retry only twice and still appear conservative, but together they can cause a single task to issue eight or nine requests.
Retry attempts should therefore be counted against a shared task identifier. Logs should include at least task_id, attempt, error_type, proxy_request_id, and response_bytes. This makes it possible to determine how many proxy requests and how much bandwidth a single business task actually consumes.
Where Should Bandwidth Optimization Begin?
Reduce unnecessary data transfer before considering a larger bandwidth or traffic plan.
Many monitoring programs need only a few fields but still download complete images, fonts, video thumbnails, and large scripts. Even a cheaper traffic plan cannot offset the long-term cost of continuously transferring content that the task never uses.
Prioritize the following four actions:
- Use lightweight endpoints. If the target provides a public API or structured data, avoid repeatedly downloading the full page.
- Limit response size. Stop reading once the required business fields have been obtained, and terminate abnormally large responses.
- Enable compression. Confirm that the client supports compressed responses and track both compressed and uncompressed byte counts.
- Separate listing and detail tasks. Let listing pages detect changes, and fetch detail pages only for new or modified records.
Soft and hard thresholds can be configured for individual responses. For example, if a normal page averages 150 KB, responses larger than 500 KB can be marked as abnormal, while responses exceeding 2 MB can be terminated and sent to a manual review queue. These thresholds should be adjusted using real samples rather than copied directly.
HEAD requests also require careful evaluation. They can sometimes reduce bandwidth consumption, but not every target supports them correctly, and some services still perform the full server-side processing. Whether they actually save money must be verified using response headers and real billing data rather than inferred from the request method alone.
How Can You Tell Whether Scaling Is Working Instead of Merely Increasing the Bill?
Scaling efficiency should be measured by cost per valid result, not by the number of requests sent.
Generate the following metrics hourly or daily:
| Metric | Calculation | Purpose |
|---|---|---|
| Business success rate | Valid results ÷ Total tasks | Determine whether tasks were completed |
| Request amplification factor | Actual requests ÷ Original tasks | Identify retries and duplicate scheduling |
| Average response size | Total response bytes ÷ Actual requests | Locate traffic inflation |
| Traffic per 1,000 valid results | Total traffic ÷ Valid results × 1,000 | Compare different operating strategies |
| Cost per 1,000 valid results | Total proxy cost ÷ Valid results × 1,000 | Decide whether further scaling is justified |
For example, suppose concurrency is increased from 20 to 40. Total requests rise by 60%, but valid results increase by only 15%, causing the cost per 1,000 valid results to rise. This is not a successful scale-up.
The next step should be to investigate queue timeouts, increased retries, duplicate tasks, or unusually large responses—not to raise concurrency again.
When comparing configurations, change only one variable at a time. First, keep the proxy product, task sample, and testing window fixed while testing concurrency levels of 10, 20, and 30. Then keep concurrency fixed and compare different timeout and retry policies.
If every parameter changes at once, the final report may look busy but will provide no reliable explanation for the outcome.
How Should Tunnel Proxy Product Limits Be Included in the Budget?
Request-rate and bandwidth limits on the proxy side should be treated as hard scheduler boundaries, not as signals to slow down only after errors appear.
According to the public product specifications for the ja.cn domestic tunnel proxy, the default capacity is 5 requests per second with 5M bandwidth, and abnormal IP addresses can be switched automatically.
This mechanism can reduce the need to maintain local proxy lists for continuous monitoring workloads. However, automatic switching addresses recovery from abnormal proxy nodes. It does not absorb unlimited business concurrency, nor does it guarantee that every request will produce a valid result.
Product limits can be stored in a central configuration service, with 20%–30% of the available capacity reserved as headroom. For example, if the request limit is 5 requests per second, the scheduler can initially operate at 3–4 requests per second. The rate can then be adjusted gradually according to response times, timeout rates, and queue backlog.
Before formally expanding capacity, test different concurrency levels against the same set of compliant monitoring targets. The purpose is to confirm whether the business success rate and cost per valid result actually improve.
What Is a Practical Sequence for Controlling Costs?
First eliminate invalid requests, then reduce per-request traffic, and only then consider expanding proxy capacity.
The following sequence can be used:
- Standardize task identifiers. Ensure that the same task can be traced across the queue, request process, and retry logs.
- Apply target-specific rate limits. Configure global concurrency, per-target concurrency, and requests-per-second limits.
- Tighten retry policies. Retry only recoverable errors and maintain a unified attempt count.
- Control response sizes. Prevent unnecessary resources from being downloaded and terminate abnormally large responses.
- Calculate cost per valid result. Measure proxy costs against usable results rather than request count.
- Scale incrementally. Increase only one concurrency tier at a time while keeping the task sample unchanged for comparison.
If the first four steps have not been completed, purchasing a larger plan will usually make inefficient tasks run faster without making them more efficient.
A healthy scale-up should satisfy three conditions at the same time: queue backlog decreases, valid-result volume increases, and cost per valid result does not deteriorate significantly.
FAQ
Q: Is the number of threads equal to tunnel proxy concurrency?
No. Threads or coroutines represent the number of client-side task executors. Actual concurrency is also affected by connection pool settings, requests-per-second limits, timeout waits, and target response speed.
Twenty coroutines may result in only five requests actively transferring data. During a concentrated retry event, the same twenty coroutines may also create a much higher instantaneous load. Actual in-flight requests and requests per second should be used as the primary measurements.
Q: If the success rate drops after increasing concurrency, should more proxy capacity be added first?
Not immediately. First inspect per-target concurrency, P95 response time, timeout categories, and the request amplification factor.
If the success rate declines while response times increase, the request rate should usually be reduced first. If proxy connections remain normal but parsing failures increase, inspect page structure and business logic instead.
Q: Is limiting requests per second enough to control costs?
It controls only part of the cost. A requests-per-second limit constrains instantaneous frequency but does not solve oversized responses, duplicate scheduling, or multilayer retries.
Two systems may both operate at five requests per second, but if one receives 100 KB per response and the other receives 1 MB, their bandwidth costs can still differ by a factor of ten.
Q: How should monitoring frequency be adjusted?
Segment targets according to how quickly their data changes and how much business value they carry.
Objects that change frequently or have a high business impact can retain a higher monitoring frequency. Objects that remain unchanged for long periods should be checked less often. When a change is detected, their frequency can be increased temporarily.
Scanning every target at a fixed frequency is simple to implement, but it usually spends a large portion of the budget on data that has not changed.
Q: When is it appropriate to purchase a larger proxy plan?
Consider scaling only after client-side rate limiting, retry behavior, and response sizes are under control and the task queue still shows persistent backlog.
A scale-up test should keep targets, time windows, and business rules consistent, then compare the increase in valid results against the additional cost. If costs grow substantially faster than valid-result volume, the workload should be optimized before more capacity is added.
