Node Unblocker: A Practical Guide to the Node.js Web Proxy

By Nicholas St. Germain —

What is Node Unblocker?

Node Unblocker is an open-source Node.js library that acts as a web proxy: you point it at a URL, and it fetches the page on your behalf, rewrites all the links, scripts, and cookies so they continue to flow through your server, and streams the result back to the client. It started life as a censorship circumvention tool - the kind of thing students used to read blocked sites at school - and grew into a general-purpose proxy library that scrapers and developers reach for when they need to transform web content rather than just forward requests.

If you have used http-proxy-middleware for an API, Node Unblocker solves a different problem. It handles the messy, browser-facing details: rewriting <a href> and <img src>, fixing the Path attribute on Set-Cookie headers, wrapping fetch and XMLHttpRequest on the client so subsequent requests stay inside the proxy, and gracefully streaming responses without buffering the whole page in memory.

This post is a practical, end-to-end guide: install it, stand up a working server, layer on the middleware you almost always need (user-agent rotation, upstream proxy, blocklists, logging), and combine it with Puppeteer for JavaScript-heavy targets. If you want the architectural deep dive - how the streaming pipeline, URL prefixer, and cookie transfer actually work under the hood - read Node Unblocker Under the Hood after this.

When to Reach for Node Unblocker

Node Unblocker is the right tool when you need to:

  • Proxy an entire browsing session through your own server with all subresources rewritten - useful for content access, internal mirrors, or proxying a third-party UI.
  • Transform HTML or CSS in flight - inject a banner, strip ads, redact selectors, swap out a logo, or normalize markup before it reaches the client.
  • Build a private scraping endpoint that combines URL rewriting with your own auth, rate limits, and upstream IP rotation.

It is not the right tool if you only need to forward JSON between a frontend and an API - http-proxy-middleware is simpler. It is also not a Cloudflare bypass or anti-bot solution; you will still need residential or ISP proxies on the egress side for protected targets.

Setting Up Node Unblocker

Installation

Spin up a fresh project and install the dependencies:

mkdir node-unblocker-demo && cd node-unblocker-demo
npm init -y
npm install express unblocker

Add Puppeteer only if you plan to drive the proxy from a headless browser:

npm install puppeteer

A Minimal Proxy Server

The smallest useful Node Unblocker server is a single Express app that mounts the middleware under a prefix:

const express = require('express');
const Unblocker = require('unblocker');

const app = express();

const unblocker = new Unblocker({
    prefix: '/proxy/',
});

app.use(unblocker);

app.get('/', (req, res) => {
    res.send(
        'Node Unblocker is running. ' +
        'Try /proxy/https://example.com/ to load a page through it.'
    );
});

const PORT = process.env.PORT || 3000;

// http.createServer is needed so Unblocker can attach a WebSocket upgrade handler
const server = app.listen(PORT, () => {
    console.log(`Server running on port ${PORT}`);
});

server.on('upgrade', unblocker.onUpgrade);

A few things to notice:

  1. The prefix is the URL namespace your proxied requests live under. Anything outside that prefix is your normal app.
  2. Wiring up server.on('upgrade', unblocker.onUpgrade) is what enables WebSocket proxying. Skip it and any site that relies on WebSockets (most modern apps) will silently break inside the proxy.
  3. Visiting http://localhost:3000/proxy/https://example.com/ should return the page with all links rewritten to stay on your server.

Driving the Proxy with Puppeteer

For JavaScript-heavy targets, point Puppeteer at the proxied URL instead of the origin and extract whatever you need:

const puppeteer = require('puppeteer');

async function scrapeViaUnblocker(targetUrl) {
    const browser = await puppeteer.launch({ headless: 'new' });
    const page = await browser.newPage();

    const proxiedUrl = `http://localhost:3000/proxy/${targetUrl}`;
    await page.goto(proxiedUrl, { waitUntil: 'networkidle2' });

    const paragraphs = await page.evaluate(() => {
        return Array.from(document.querySelectorAll('p')).map(
            (p) => p.textContent.trim()
        );
    });

    await browser.close();
    return paragraphs;
}

scrapeViaUnblocker('https://example.com/').then(console.log);

Because Node Unblocker injects a client-side wrapper around fetch and XMLHttpRequest, AJAX calls that the page makes after load will also route through the proxy automatically - you do not need to instrument them yourself.

Custom Middleware You Will Almost Always Want

Node Unblocker exposes requestMiddleware and responseMiddleware arrays. Each function gets a data object with the URL, headers, and streams, and can short-circuit the pipeline by responding directly. A handful of additions are worth adding to almost any deployment.

Rotating User Agents

Static User-Agent strings are an easy fingerprint. Rotate from a small pool:

const userAgents = [
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ' +
        '(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36',
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 ' +
        '(KHTML, like Gecko) Version/17.4 Safari/605.1.15',
    'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 ' +
        '(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36',
];

function rotateUserAgent(data) {
    data.headers['user-agent'] =
        userAgents[Math.floor(Math.random() * userAgents.length)];
}

const unblocker = new Unblocker({
    prefix: '/proxy/',
    requestMiddleware: [rotateUserAgent],
});

Blocking Internal Network Targets

Without an SSRF guard, an open Node Unblocker instance can be turned into a port scanner against your own infrastructure. Reject private and loopback hosts before the request goes out:

function blockInternal(data) {
    const { hostname } = new URL(data.url);
    const isInternal =
        hostname === 'localhost' ||
        hostname.startsWith('127.') ||
        hostname.startsWith('10.') ||
        hostname.startsWith('192.168.') ||
        /^172\.(1[6-9]|2[0-9]|3[0-1])\./.test(hostname);

    if (isInternal) {
        data.clientResponse.status(403).send('Blocked');
    }
}

If a middleware sends a response on clientResponse, the rest of the pipeline short-circuits - no remote request is made.

Routing Outbound Through an Upstream Proxy

Node Unblocker uses your server's IP for outbound requests by default, which is a problem for any target with even mild bot detection. Attaching an https-proxy-agent swaps the egress IP per request:

const { HttpsProxyAgent } = require('https-proxy-agent');

const upstream = new HttpsProxyAgent(
    'http://USERNAME:PASSWORD@proxy.statproxies.com:3128'
);

function useUpstreamProxy(data) {
    data.agent = upstream;
}

const unblocker = new Unblocker({
    prefix: '/proxy/',
    requestMiddleware: [useUpstreamProxy],
});

For real rotation, swap a single agent for a pool indexed per request - see the scaling section of the deep dive for a full example.

Handling Common Scraping Challenges

Waiting for Dynamic Content

When a page hydrates client-side, do not rely on load. Wait for a selector that only appears once the data is rendered:

await page.goto(proxiedUrl, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.results-grid', { timeout: 10000 });

const items = await page.$$eval('.results-grid .item', (nodes) =>
    nodes.map((n) => n.innerText.trim())
);

Pagination

Most paginated lists either expose a query parameter or a "next" link. Prefer the query parameter - it is far more reliable than clicking through the DOM:

async function scrapeAllPages(baseUrl, maxPages = 10) {
    const all = [];
    for (let pageNum = 1; pageNum <= maxPages; pageNum++) {
        const url = `http://localhost:3000/proxy/${baseUrl}?page=${pageNum}`;
        await page.goto(url, { waitUntil: 'networkidle2' });

        const items = await page.$$eval('.item', (nodes) =>
            nodes.map((n) => n.innerText)
        );
        if (items.length === 0) break;

        all.push(...items);
    }
    return all;
}

Things That Will Not Work Through the Proxy

It is faster to know the limits up front than to debug them later:

  • OAuth and postMessage flows - sign-in-with-Google and similar redirect-heavy flows do not survive URL rewriting.
  • Heavily-scripted SPAs like Discord, X, YouTube, or Instagram tend to break in subtle ways even with the client-side wrapper.
  • Anti-bot challenges (Cloudflare, PerimeterX, DataDome) - Node Unblocker does not solve them. You need a managed unblocking service or a real browser with stealth patches.
  • Non-rewritten content types - by default only text/html, text/css, and the XHTML variants are processed. Images, fonts, and JSON pass through untouched.

Pairing Node Unblocker with Stat Proxies

Node Unblocker handles the content layer well - URL rewriting, cookies, headers, streaming - but the IP it presents to target sites is whatever your server happens to be running on. That is fine for a private mirror or internal tool, but for any kind of scraping at scale you want trusted residential or ISP IPs on the egress side.

The pattern is straightforward: keep Node Unblocker for everything client-facing, and route its outbound HTTP/HTTPS through Stat Proxies via an https-proxy-agent. You get clean URL rewriting from Node Unblocker, IP diversity from the proxy pool, and a single layer your application has to talk to. For very high-throughput deployments, also configure connection pooling on the agents and run multiple Node Unblocker instances behind a load balancer - both are covered in the architecture deep dive.

Wrapping Up

Node Unblocker is a small library that does a surprisingly large amount of work: it streams, rewrites, and stitches together a usable proxied browsing experience with a few lines of Express. For most scraping use cases, the pattern that works is the one in this post - a minimal server, a few targeted middleware functions, an upstream proxy for IP diversity, and Puppeteer (or Playwright) when the target needs a real browser. Once you have that running, the under-the-hood guide is the next step if you want to scale it or extend it with custom middleware.