I run this in production in Snagbox, my open-source NestJS downloader API. For this post I rebuilt its SSE layer in a small test app on NestJS 12.1.3 and Node.js 22.20, drove it from headless Chrome 155, and measured what each design choice actually does. Below are the patterns I'd keep, the ones I changed, and the numbers behind them.

TL;DR: NestJS SSE in production

Return an Observable<MessageEvent> from an @Sse() route. Then:

• Give each job its own ReplaySubject(1), so a client that connects mid-job gets the latest progress at once. In my test that took 2 to 4 ms, against 451 to 1,002 ms with a plain Subject.
• Set id on every message yourself. Nest numbers messages per connection, so after a reconnect the ids start at 1 again and Last-Event-ID is useless.
• Close the EventSource on the client when the job finishes. When your Observable completes, the browser treats it as a dropped connection and reconnects, and it replayed my stream three times in 1.6 seconds.
• Send a heartbeat as a comment frame ({ comment: "ping" }). The client never sees it, and Nest gives it no id.
• Don't open one stream per item in a batch. Over HTTP/1.1, Chrome opened 6 of 8 streams to my server, and an ordinary fetch() to the same host was still waiting 5 seconds later. Poll one batch endpoint instead, or serve over HTTP/2.

What are Server-Sent Events?

SSE is a plain HTTP response with Content-Type: text/event-stream that never ends until one side closes it. The server writes UTF-8 frames separated by a blank line. Each frame can carry data:, an event: name, an id: and a retry: delay, and a line starting with : is a comment. The format and the reconnection rules are in the WHATWG HTML spec.

In the browser, EventSource does all the client work. It parses frames, fires events, remembers the last id, and reconnects after a dropped connection, sending that id back in a Last-Event-ID header.

SSE only goes from the server to the client. For job progress, notifications, build logs and streamed LLM tokens, that is all you need. The client can still send commands through normal POST requests.

SSE vs WebSockets vs polling

Use WebSockets when the client also streams data back at a high rate, for example in chat or collaborative editing. Use polling when updates are rare or when you track many items at once (more on that below). For one long job and one watcher, SSE is the best fit.

How to use @Sse in NestJS

The smallest working endpoint returns an Observable. Nest writes every emitted value as a frame and ends the response when the Observable completes.

import { Controller, MessageEvent, Sse } from "@nestjs/common";
import { Observable, interval, map, take } from "rxjs";

@Controller("demo")
export class DemoController {
  @Sse("ticks")
  ticks(): Observable<MessageEvent> {
    return interval(1000).pipe(
      take(5),
      map((i) => ({ data: { tick: i } })),
    );
  }
}

Objects in data are serialized as JSON. The MessageEvent interface in @nestjs/common 12.1.3 has five optional fields: data, id, type, retry and comment. The full list of features is in the NestJS SSE docs.

I checked the response headers with curl -D -. Nest 12.1.3 sends Content-Type: text/event-stream, Cache-Control: private, no-cache, no-store, must-revalidate, max-age=0, no-transform, Connection: keep-alive and X-Accel-Buffering: no. The last one tells Nginx not to buffer the stream, which is the most common reason SSE "works locally but not in production". You don't have to set it yourself.

Streaming job progress: one ReplaySubject per job

A real job runs in a worker, not in the controller. The worker needs somewhere to publish progress, and the SSE route needs somewhere to subscribe. In Snagbox that is a small service holding one RxJS subject per job (source). This is a trimmed version:

import { Injectable } from "@nestjs/common";
import { Observable, ReplaySubject } from "rxjs";

export type JobEvent =
  | { type: "progress"; seq: number; percentage: number }
  | { type: "complete"; seq: number; url: string }
  | { type: "error"; seq: number; message: string };

@Injectable()
export class JobEventsService {
  private readonly channels = new Map<string, ReplaySubject<JobEvent>>();

  private channel(jobId: string): ReplaySubject<JobEvent> {
    let subject = this.channels.get(jobId);
    if (!subject) {
      // Buffer of 1: a client that joins mid-job gets the latest frame at once.
      subject = new ReplaySubject<JobEvent>(1);
      this.channels.set(jobId, subject);
    }
    return subject;
  }

  emit(jobId: string, event: JobEvent): void {
    this.channel(jobId).next(event);
  }

  finish(jobId: string, event: JobEvent): void {
    const subject = this.channel(jobId);
    subject.next(event);
    subject.complete();
    // Late subscribers still get the terminal frame for a while, then we free it.
    setTimeout(() => this.channels.delete(jobId), 5 * 60_000).unref();
  }

  stream(jobId: string): Observable<JobEvent> {
    return this.channel(jobId).asObservable();
  }
}

Why ReplaySubject(1) and not Subject? Users don't open the progress page at the exact moment a job starts. They refresh, switch tabs, or open it from a notification. A plain Subject only delivers future values, so a late client sees an empty progress bar until the next update arrives.

I measured this. A job emitted progress every 2 seconds, and a client connected between two updates, 5 seconds in, five times. With ReplaySubject(1) the first frame arrived in 2 to 4 ms. With Subject it took 451 to 1,002 ms, which is simply the wait for the next emit. With slower jobs, such as a big download that reports every few seconds, that wait gets longer.

Keep the buffer at 1. Progress is a state, not a log, so the latest frame is all a new client needs, and a bigger buffer only replays stale percentages.

This map lives in one process. If you run several API instances, the worker's events have to reach whichever instance holds the client's connection. Put a broker in between, such as Redis pub/sub, and feed each instance's subjects from it.

The production SSE controller

Here is the route, close to Snagbox's downloads controller but with three changes I made after testing:

import { Controller, MessageEvent, NotFoundException, Param, Sse } from "@nestjs/common";
import { Observable, endWith, ignoreElements, interval, map, merge, of, share, takeUntil } from "rxjs";
import { JobEventsService, type JobEvent } from "./job-events.service.js";
import { JobStore } from "./job-store.js";

const HEARTBEAT_MS = 15_000;
const toMessage = (event: JobEvent): MessageEvent => ({ id: String(event.seq), data: event });

@Controller("jobs")
export class JobsController {
  constructor(
    private readonly events: JobEventsService,
    private readonly store: JobStore,
  ) {}

  @Sse(":id/events")
  stream(@Param("id") id: string): Observable<MessageEvent> {
    const job = this.store.get(id);
    if (!job) throw new NotFoundException(`No job ${id}`);

    // Finished before the client connected: send the terminal frame and stop.
    if (job.last && job.last.type !== "progress") return of(toMessage(job.last));

    const events$ = this.events.stream(id).pipe(map(toMessage), share());
    const done$ = events$.pipe(ignoreElements(), endWith(null));
    // A comment frame (": ping") keeps proxies from reaping the idle socket.
    const heartbeat$ = interval(HEARTBEAT_MS).pipe(
      map((): MessageEvent => ({ comment: "ping" })),
      takeUntil(done$),
    );
    return merge(events$, heartbeat$);
  }
}

JobStore is wherever you keep job state. Snagbox keeps its download records in Postgres through Prisma. I ran this exact controller and checked each branch with curl:

• An unknown id returns a normal 404 JSON response. Throwing before you return the Observable works, because Nest hasn't sent the stream headers yet. Under the spec, a non-200 response makes EventSource give up rather than retry, which is what you want for a job that doesn't exist.
• A finished job returns one complete frame and ends.
• A running job streams frames with id: 6, id: 7 and so on, matching the job's own sequence number, with : ping comments in between.

Why set the id yourself

If a message has no id, Nest 12.1.3 adds one: a counter that starts at 1 on each connection. I tested what that means for reconnects in headless Chrome. The browser reconnected and sent Last-Event-ID: 3, and the new connection numbered its frames from 1 again. The header is there, but it no longer means anything.

Using the job's sequence number as the id fixes that. On reconnect, read the header with @Headers("last-event-id") and skip events the client has already seen. For progress, the ReplaySubject already sends the latest state, so you can often ignore the header and still be correct.

The heartbeat: a comment, not a data frame

Snagbox merges a heartbeat into the stream every 15 seconds. The comment in its source says why: proxies reap idle connections, and the Next.js rewrite proxy in front of the API is one of them. takeUntil(done$) stops the timer when the job ends, so the heartbeat never holds a finished stream open.

Snagbox sends the heartbeat as data: {"type":"ping"} and every client has to ignore it. On Nest 12.1.3 you can return { comment: "ping" } instead. Nest writes it as : ping, gives it no id, and EventSource doesn't fire any event for it. I confirmed both in the curl output and in Chrome.

The share() matters too. Both merge and done$ subscribe to events$, and share() makes them use one subscription to the job channel instead of two.

The client: close the EventSource when the job ends

This is the mistake I'd most expect to reach production. When the job finishes, the Observable completes and Nest ends the response. To EventSource, an ended response is a dropped connection, so it waits for the retry delay and connects again.

I tested this with an endpoint that sent three events and completed, with retry: 300. In 1.6 seconds Chrome received the same three events three times, plus the first event of a fourth run. For a progress bar that means flicker and wasted requests. For "show a toast when the download is ready", it means the toast fires again and again.

The fix is on the client: treat the terminal event as the end.

type JobEvent =
  | { type: "progress"; seq: number; percentage: number }
  | { type: "complete"; seq: number; url: string }
  | { type: "error"; seq: number; message: string };

export function watchJob(jobId: string, onEvent: (event: JobEvent) => void) {
  const source = new EventSource(`/api/jobs/${jobId}/events`, { withCredentials: true });

  source.onmessage = (message) => {
    const event = JSON.parse(message.data) as JobEvent;
    onEvent(event);
    if (event.type === "complete" || event.type === "error") source.close();
  };

  // Network blips are retried by the browser. A 404 or 401 closes the stream.
  source.onerror = () => {
    if (source.readyState === EventSource.CLOSED) onEvent({ type: "error", seq: -1, message: "stream closed" });
  };

  return () => source.close();
}

Two more things I checked in Chrome:

• Named events skip onmessage. If you set type: "progress" on the Nest MessageEvent, the frame goes out as event: progress, and only addEventListener("progress", ...) receives it. In my test, onmessage got the unnamed frame and missed the named one. Either put the type inside data, as above, or add a listener for every event name you send.
• EventSource can't send custom headers. The constructor takes a URL and a withCredentials flag, nothing else. If your API uses an X-API-Key or a bearer token, use a cookie session for the browser, a short-lived signed token in the query string, or fetch() with a streamed response body and your own frame parser.

Don't open one stream per item: the six-connection limit

Over HTTP/1.1, browsers allow only six connections per host, and each open EventSource holds one for as long as it lives. Snagbox's controller has a comment about this: batch downloads poll GET /v2/downloads?ids=... instead of opening one stream per job, because "a 50-item batch would stall on its own event streams".

I measured how bad it is. A page in headless Chrome 155 opened 8 EventSources to my local Nest server over plain HTTP/1.1. Six opened within 5 ms. The other two never opened during the 3-second window. Then the page made an ordinary fetch() to the same server, and it was still pending after 5 seconds. Six progress streams can freeze the rest of your app's API calls on that host.

You have three ways out:

1. Serve over HTTP/2. All requests share one connection, and the limit becomes the number of concurrent streams the server allows. MDN's EventSource page notes that this defaults to 100. If a proxy or CDN in front of your API already speaks HTTP/2 to the browser, you may have this for free.
2. Multiplex. Open one stream per user or per batch and include the job id in every event.
3. Poll a batch endpoint for many items. This is what Snagbox does for playlists:

@Get()
getMany(@Query("ids") ids = "") {
  return ids
    .split(",")
    .map((id) => id.trim())
    .filter(Boolean)
    .slice(0, 100)
    .map((id) => this.store.get(id) ?? { id, status: "unknown" });
}

One request every second or two for a 50-item batch costs far less than 50 streams, and it can't block anything.

SSE vs polling: what one job costs

I ran one simulated job that lasted 10 seconds and emitted 40 progress updates, one every 250 ms. A Node.js client watched it three ways, and I counted the bytes from the socket itself (socket.bytesRead and socket.bytesWritten), so HTTP headers are included. The client and server ran on the same box, over localhost, with the Express adapter.

Polling once a second used about the same bytes as SSE but showed only a quarter of the updates, so the bar moves in jumps and can show a value up to a second old. Polling fast enough to see every update took 41 requests and 3.7 times the bytes. SSE delivered each update 0.8 ms after the server emitted it on average, with a 2 ms maximum.

Each SSE update cost about 110 bytes, which is mostly the JSON. Each polled response carried around 320 bytes, because every one repeats the HTTP headers. With API-key auth, a rate limiter and logging on every request, the request count matters more than the bytes.

How much memory does an open SSE stream use?

An open stream is a socket, a response object and an RxJS subscription. I opened 1,000 and then 5,000 more concurrent streams against the same job and read the server's resident memory from /proc/<pid>/status. The client ran in a separate process.

The per-stream figure fell at the larger count, probably because the first batch also paid one-off costs as the heap grew (RSS didn't shrink between the runs). Treat it as a rough range for Nest on Express and Node 22, not a constant. It does show that one instance can hold thousands of idle progress streams. Your file-descriptor limit (ulimit -n) and proxy timeouts are more likely to stop you first.

I also counted live subscriptions in the service. With 5,000 streams open, the count was 5,000. 1.5 seconds after the clients disconnected, it was 0. Nest unsubscribes from your Observable when the client goes away, so the RxJS teardown runs and the subject doesn't leak subscribers.

If your handler does async setup before it returns the Observable, such as opening a database cursor or an LLM session, Nest 12.1.3 also has an @SseSignal() parameter decorator. It injects an AbortSignal that fires when the stream ends, so you can release setup resources even if the client left before the Observable existed.

A smooth progress bar: never let it go backwards

One Snagbox detail isn't about SSE but shows up in every progress UI. Snagbox runs yt-dlp, which downloads video and audio as two separate passes and reports each from 0 to 100%. Forwarded as-is, the bar fills, snaps back to zero and fills again.

Snagbox's ProgressAggregator fixes this. When downloaded bytes drop, it banks the finished pass. It keeps the percentage monotonic and caps it at 99.9 while the process is alive, because the last pass finishing isn't the job finishing, since the merge still has to run. Only the complete event reports 100%. If your job has more than one phase, do the same before anything reaches the stream.

How I tested

• Box: Linux, 8 vCPUs, shared with other workloads. Node.js 22.20.0, @nestjs/core 12.1.3 on @nestjs/platform-express 12.1.3, RxJS 7.8.2, TypeScript 5.9, compiled to ESM.
• Jobs were simulated with setInterval, so the tests measure the transport and Nest, not a real download.
• Bytes came from the client socket's counters, with keep-alive for the polling client.
• The late-joiner test ran five times per subject type, joining 5.0 to 5.5 s into a job that emitted every 2 s.
• Browser tests used puppeteer-core driving Chrome 155 headless, against http://127.0.0.1 over HTTP/1.1.
• Memory is VmRSS of the Nest process, read before opening streams and 4 seconds after the last one opened.

These are single-machine, localhost numbers. Over a real network, polling gets worse, because each request pays the round trip, while SSE pays it once.

Common mistakes with NestJS SSE

• Not closing the EventSource on the terminal event. The browser reconnects and replays the job.
• Relying on Nest's automatic ids. They restart at 1 on every connection. Use your own sequence.
• Sending type on the MessageEvent and listening with onmessage. Named events only reach addEventListener.
• Using a plain Subject. Late clients wait for the next update. Use ReplaySubject(1).
• One stream per list item over HTTP/1.1. Six streams block the host. Poll a batch endpoint or use HTTP/2.
• No heartbeat behind a proxy. Idle streams get cut. Send a comment frame every 15 to 30 seconds.
• Putting the API key in a header the browser can't send. EventSource has no headers option.
• Keeping job channels forever. Delete them a few minutes after the terminal event, as Snagbox does.

For the queue side of long-running work, read Building an AI automation pipeline in Node.js with BullMQ. If you're on an older Nest version, the NestJS 12 migration guide covers the upgrade. For a secure API base to put this in, see my NestJS + Prisma API starter.

Key takeaways

• NestJS streams SSE from any @Sse() route that returns an Observable, and sets text/event-stream, no-cache and X-Accel-Buffering: no for you.
• A ReplaySubject(1) per job gave late clients their first frame in 2 to 4 ms, against up to 1,002 ms with a plain Subject.
• Set id from your job's sequence number. Nest's automatic ids restart at 1 on every reconnect.
• Close the EventSource on the terminal event. A completed stream made Chrome reconnect and replay it three times in 1.6 s.
• Send heartbeats as { comment: "ping" }, which the browser ignores and Nest leaves without an id.
• Over HTTP/1.1, Chrome opened 6 of 8 streams and then stalled a normal fetch(). Use HTTP/2, one multiplexed stream, or a batch polling endpoint.
• For one 10-second job, SSE used 1 request and 4,516 bytes. Polling every 250 ms needed 41 requests and 16,506 bytes for the same updates.

Originally published on fabwaseem.com.

Prisma 8 is a rewrite of Prisma ORM, not a version bump. The schema becomes a contract, prisma generate becomes prisma contract emit, @prisma/client is replaced by @prisma/orm-postgres, and queries are chained (db.orm.public.User.where(...).all()). It is still a release candidate, but since August 2026 npm install prisma installs it.

That last fact is the urgent part. Any Prisma 7 project, CI job or Dockerfile that installs prisma without a version now gets a CLI with no generate, migrate or db push. I upgraded a real Prisma 7 app on PostgreSQL 17 with the official side-by-side path, and this guide gives the commands, the output, two gotchas the docs don't spell out, and a clear answer on whether to move now. Facts are cited to the Prisma ORM release status page.

What is Prisma 8?

Prisma 8 (Prisma ORM 8) is the next major version of Prisma ORM, a typed data layer for TypeScript. You describe models in a contract file, prisma contract emit writes contract.json and contract.d.ts, and you query through a db client from @prisma/orm-postgres. There is no generated PrismaClient, and migrations are TypeScript files you can review.

Key facts on October 17, 2026:

The CLI and the library are versioned separately, so prisma 8.0.0-rc.23 and @prisma/orm-postgres 8.0.0-rc.18 are a matching pair. Prisma says "any two latest versions work together."

Is Prisma 8 stable? When is the release date?

No, not yet. Prisma 8 is a release candidate, and Prisma's release status page says general availability "is expected in October 2026." On October 17, 2026 the newest builds were prisma 8.0.0-rc.23 (published October 15) and @prisma/orm-postgres 8.0.0-rc.18 (October 14). Prisma warns that "some details of the API may still change."

Two support dates follow from the release:

• Prisma ORM 7 keeps receiving bug fixes and security updates "for 18 months from the day Prisma ORM 8 reaches general availability."
• Prisma ORM 6 receives security patches only until November 19, 2026.

So there is no rush to leave Prisma 7, but Prisma 6 projects should move to 7 now.

Why does npm install prisma install Prisma 8?

Because Prisma moved the npm latest tag to Prisma 8 while it is still an RC. Prisma's August 28 changelog says the release "makes Prisma 8 available through prisma@latest." Prisma 7 is now prisma@prev. A Prisma 7 project that installs prisma without a version range gets a CLI that cannot read its workflow.

I ran the old commands with the Prisma 8 CLI in a Prisma 7 project:

$ npx prisma generate
✘ [CLI.UNKNOWN_COMMAND] No command registered for `generate`
$ npx prisma migrate deploy
✘ [CLI.UNKNOWN_COMMAND] No command registered for `migrate`, did you mean `migration`?
$ npx prisma db push
✘ [CLI.UNKNOWN_COMMAND] No command registered for `push`, did you mean `update`?

Do this today, even if you never upgrade: pin both packages, then grep your CI, Dockerfiles and scripts for npx prisma and prisma@latest.

{
  "devDependencies": { "prisma": "^7" },
  "dependencies": { "@prisma/client": "^7" }
}

Anywhere a command runs without a lockfile, write npx prisma@7 migrate deploy instead of npx prisma migrate deploy. If you build images with a multi-stage Dockerfile like the one in my Next.js Dockerfile guide, the npm ci step respects the lockfile; a separate npx prisma ... line in a later stage does not.

What changed in Prisma 8?

Almost everything you type. The schema is a contract, codegen is contract emit, the client is built in src/prisma/db.ts, queries are chained, migrations are TypeScript packages tracked by a hash marker in the database, and several Prisma 7 features ($extends, nested updates, increment, JSON filters, P2002 codes) are missing. Your database tables and data do not change.

Commands

Queries

public is the PostgreSQL schema the table lives in. The full mapping is in Prisma's Coming from Prisma ORM 7 reference.

Schema types

In a hand-written contract, @db. attributes become the field type (String @db.VarChar(255) is VarChar(255)), Json must be written Jsonb, @updatedAt is temporal.updatedAt(), cuid() is cuid(2), and implicit many-to-many relations need an explicit join model. DateTime values come back as Temporal.Instant, which Node.js 22 and 24 do not have built in. On my Node.js 24.22.0, node -p "typeof Temporal" printed undefined, so a hand-written contract needs temporal-polyfill.

What is missing in Prisma 8?

Prisma 8 does not yet have several Prisma 7 features. The release status page lists $extends, "filtering inside JSON columns, atomic increment, most nested writes, transaction isolation levels, and the P2002-style error codes." $extends will not return; middleware, custom collections and scopes replace it. If your app relies on these, stay on Prisma 7 for now.

I replaced an increment with the documented raw SQL builder, and it ran on 8.0.0-rc.18:

import { db } from "./prisma/db.ts";

const post = db.sql.public.Post;
const bump = db.raw.sql`UPDATE "Post" SET views = views + 1 WHERE id = ${id} RETURNING id, views`
  .returnsRow({ id: post.columns.id, views: post.columns.views })
  .build();

const rows = await db.runtime().query(bump); // [ { id: 1, views: 3 } ]

How do I migrate from Prisma 7 to Prisma 8?

Run both versions side by side on the same database. One command (npx prisma@latest orm init --from-prisma7-schema prisma/schema.prisma) renames Prisma 7 to prisma7, installs Prisma 8, and lets it read your existing schema.prisma. You then sign the database, move routes one at a time, and hand migrations to Prisma 8 only when you choose.

This is the path from Prisma's official Prisma ORM 7 to 8 guide, and the prisma8-and-7-example repo shows each phase. My test app was a Hono API on Prisma 7.10.1 with @prisma/adapter-pg, PostgreSQL 17.11, Node.js 24.22.0, a Role enum, a DateTime @default(now()), a Json? column and an increment update.

Step 1: Run orm init

npx prisma@latest orm init --from-prisma7-schema prisma/schema.prisma

Write prisma@latest in full, because inside a Prisma 7 project npx prisma still runs version 7. In CI or from a coding agent, the command stops and asks for consent:

CLI.CONSENT_REQUIRED: "Prisma 7 is installed as `prisma`. Keep it as @prisma/prisma7
(binary prisma7) and move `prisma` to Prisma 8, and rename prisma.config.ts to
prisma7.config.ts?" requires explicit consent, and the session is not interactive.
Grant it by passing --confirm app.

Note that it had already added @prisma/orm-postgres to package.json before it stopped. With --confirm app it finished and changed exactly this:

   "dependencies": {
     "@prisma/adapter-pg": "^7.10.1",
     "@prisma/client": "^7.10.1",
+    "@prisma/orm-postgres": "^8.0.0-rc.18",
   },
   "devDependencies": {
+    "@prisma/cli-engine": "^0.7.0",
+    "@prisma/prisma7": "^7.10.1",
-    "prisma": "^7.10.1",
+    "prisma": "^8.0.0-rc.23",
   }

It renamed prisma.config.ts to prisma7.config.ts (now importing @prisma/prisma7/config), wrote a new prisma.config.ts, and emitted the client into src/prisma/:

// prisma.config.ts (written by orm init)
import "dotenv/config";
import { definePrismaConfig } from "@prisma/cli-engine";
import { defineConfig as ormConfig, prisma7Schema } from "@prisma/orm-postgres/config";

export default definePrismaConfig({
  orm: ormConfig({
    contract: prisma7Schema("prisma/schema.prisma"),
    output: "src/prisma",
    db: { connection: process.env["DATABASE_URL"]! },
  }),
});

npx prisma --version now reports 8.0.0-rc.23, and npx prisma7 generate still writes the Prisma 7 client. Update every script and CI step that must keep using Prisma 7 to call prisma7.

Step 2: Sign the database

$ npx prisma db sign
✔ app: signed b3b45c6c5fd8… (no marker before)
✔ Database signed
✔ Advanced ref "db" → b3b45c6c5fd8…

$ npx prisma db verify
✔ Database marker and schema match contract

db sign changes no tables. It adds a prisma_contract.marker table that records which contract hash the database satisfies. Prisma 7 still owns migrations, so after every prisma7 migrate dev or prisma7 migrate deploy, run npx prisma contract emit and npx prisma db sign again.

Step 3: Move one route

// src/routes/users.ts — now on Prisma 8
import { db } from "../prisma/db.ts";

users.get("/", async (c) => {
  const result = await db.orm.public.User.include("posts", (posts) =>
    posts.orderBy((post) => post.id.asc()),
  )
    .orderBy((user) => user.id.asc())
    .all();
  return c.json(result);
});

users.post("/", async (c) => {
  const body = await c.req.json<{ email: string; name?: string }>();
  try {
    return c.json(await db.orm.public.User.create(body), 201);
  } catch (error) {
    // No P2002 in Prisma 8: check the PostgreSQL SQL state instead
    if (error instanceof Error && "sqlState" in error && error.sqlState === "23505") {
      return c.json({ error: "email already exists" }, 409);
    }
    throw error;
  }
});

The posts route stayed on Prisma 7, including its increment. A post written through Prisma 7 appeared in the next Prisma 8 read, and a duplicate email returned my 409. tsc --noEmit passed on TypeScript 6.0.3 and, as a bonus, on TypeScript 7 7.0.2.

Step 4: Hand migrations to Prisma 8

When you are ready, convert the Prisma 7 schema into a real contract, point the config at it, sign, then plan a change:

npx prisma contract print --output prisma8/contract.prisma
# set contract: "prisma8/contract.prisma" in prisma.config.ts
npx prisma contract emit
npx prisma db sign

The printed contract produced the same hash as the Prisma 7 schema (unchanged, already signed), so nothing drifted. It mapped my types like this:

// use prisma-8
namespace public {
  model User {
    id        Int                @id @default(autoincrement())
    email     String
    name      String?
    role      pg.enum(Role)      @default("USER")
    createdAt TimestampString(3) @default(now())
    posts     Post[]

    @@index([email], map: "User_email_key", unique: true)
  }

  native_enum Role {
    USER = "USER"
    ADMIN = "ADMIN"
  }
}

Then I added bio String? and ran the new migration loop:

$ npx prisma contract emit
$ npx prisma migration plan --name add_user_bio
→ Review migrations/app/20261017T1408_baseline and migrations/app/20261017T1409_add_user_bio
$ npx prisma db migrate --advance-ref db
✔ Applied 1 migration(s) (1 operation(s)) across 1 contract space(s)
├─ Add column "bio" to "User"
$ npx prisma db verify
✔ Database marker and schema match contract

The first plan writes a baseline (the schema Prisma 7 built) plus your change. db migrate skips the baseline because the marker already records it. Each migration is a reviewable migration.ts:

override get operations() {
  return [
    this.addColumn({
      schema: "public",
      table: "User",
      column: col("bio", "text", { codecRef: { codecId: "pg/text@1" } }),
    }),
  ];
}

The Prisma 7 routes kept working after the new column. From here, retire prisma7 migrate from your scripts, move the remaining routes, and finally remove @prisma/prisma7, @prisma/client and @prisma/adapter-pg.

What broke when I ran Prisma 7 and 8 side by side?

Two things the guide doesn't warn about. First, orm init needs --confirm app in any non-interactive shell (CI, Docker, coding agents). Second, createdAt timestamps written by Prisma 8 were 5 hours off. Prisma 7 fills @default(now()) in the client in UTC, but Prisma 8 leaves it to the database, which used my server's local time zone.

The evidence, from the same User table:

id | createdAt                 written by
 1 | 2026-10-17 14:06:44.701   Prisma 7 (UTC)
 2 | 2026-10-17 19:08:52.282   Prisma 8 (server time zone Asia/Karachi, UTC+5)
 4 | 2026-10-17 14:09:08.795   Prisma 7 (UTC)

Prisma 7 created the column as TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, which is timestamp without time zone. Prisma's own mapping table says @default(now()) in Prisma 8 "is a database default." For a timestamp column, PostgreSQL then writes the time in the session's TimeZone setting, which defaults to the server's. After ALTER DATABASE p8demo SET timezone TO 'UTC', Prisma 8 wrote 14:09:20 at 14:09:20 UTC.

Many managed PostgreSQL services default to UTC, so you may never see this. Check before mixing writers:

SHOW timezone;  -- must be UTC (or Etc/UTC) while Prisma 7 and 8 both write

Also note the data type change: read through prisma7Schema(...), DateTime comes back as a string such as "2026-10-17 14:06:44.701", not a Date. My JSON responses changed format between the two routes (2026-10-17T14:06:44.701Z from Prisma 7), so API clients that parse dates should be tested.

Should I upgrade to Prisma 8 now?

Not for production apps yet. Pin Prisma 7 today, because that protects you from the latest tag change, and plan the move after GA. Start with Prisma 8 now only for new PostgreSQL projects that don't need the missing features, or to trial it on one route of an existing app with the side-by-side setup.

What happens to Prisma Accelerate?

Prisma retires Accelerate on December 1, 2026, according to its October 2 changelog. If your connection string starts with prisma+postgres://accelerate.prisma-data.net, switch to a pooled connection or the serverless driver before then. Accelerate's query caching has no replacement, so remove cacheStrategy from your queries first.

The same changelog sets a second deadline: Prisma Postgres Early Access databases, the ones still on PostgreSQL 16, "are deleted on November 1, 2026 at 00:00 UTC." Move each one to a new database before then. Both deadlines apply whether you stay on Prisma 7 or move to 8.

Prisma 8 upgrade checklist

Pin Prisma 7 first, then handle the deadlines, then migrate on your own schedule. These steps follow the official guide plus what I hit in testing.

1. Pin prisma and @prisma/client to ^7 and replace every unpinned npx prisma in CI and Dockerfiles with npx prisma@7 or a lockfile install.
2. Leave Accelerate before December 1, 2026, and move Early Access Prisma Postgres databases before November 1.
3. Check requirements: Node.js 22.18+, 24.11+ or 26+, TypeScript 5.9+ with strict, PostgreSQL.
4. Audit for missing features: $extends, increment, nested updates, JSON filters, isolation levels, P2002 handling.
5. Check the database time zone with SHOW timezone; and set it to UTC if Prisma 7 and 8 will both write.
6. Run npx prisma@latest orm init --from-prisma7-schema prisma/schema.prisma (add --confirm app in CI).
7. Run npx prisma db sign, then move one low-risk route to db.orm.
8. After each prisma7 migration, run prisma contract emit and prisma db sign.
9. Transfer migrations with contract print, db sign, migration plan, db migrate --advance-ref db.
10. Remove Prisma 7 when nothing imports @prisma/client.

Common Prisma 8 mistakes

The most common mistakes are letting prisma@latest slip into a Prisma 7 build, running Prisma 8's db init or db update against a database Prisma 7 already manages, forgetting to re-sign after a Prisma 7 migration, and assuming DateTime and error handling behave the same in both clients.

Related: I benchmarked Prisma 8 RC against Prisma 7 and Drizzle in Drizzle vs Prisma in 2026: Prisma 8 RC had the slowest bundled cold start (about 370 ms). If you use Better Auth, note that Better Auth 1.7 supports Prisma 5 to 7 but not Prisma 8 yet; see my Better Auth with Next.js 16 setup.

Related: Prisma's @default(uuid(7)) generates IDs in the client and sets no database default. On PostgreSQL 18 you can use dbgenerated("uuidv7()") instead; see my UUIDv7 in PostgreSQL 18 benchmark.

Key takeaways

• Prisma 8 is a release candidate (prisma 8.0.0-rc.23, @prisma/orm-postgres 8.0.0-rc.18 on October 17, 2026) with GA expected in October 2026, but npm's latest tag already installs it.
• Pin prisma and @prisma/client to ^7 today; the Prisma 8 CLI has no generate, migrate or db push.
• Prisma 8 replaces schema.prisma with a contract, prisma generate with prisma contract emit, and @prisma/client with @prisma/orm-postgres and chained queries.
• $extends, increment, most nested writes, JSON filters, isolation levels and P2002 codes are missing, and only PostgreSQL is fully supported.
• orm init --from-prisma7-schema runs Prisma 7 and 8 side by side on one database; it needs --confirm app outside an interactive terminal.
• Check SHOW timezone before mixing writers: Prisma 8 leaves now() to the database, which skewed my timestamps by 5 hours.
• Prisma 7 is supported for 18 months after Prisma 8 GA, Prisma 6 security patches end November 19, 2026, and Accelerate retires December 1, 2026.

Originally published on fabwaseem.com.

TypeScript 7 is the native, Go-based rewrite of the TypeScript compiler, and it has been the latest version on npm since July 8, 2026. It type-checks the same code as TypeScript 6 but much faster. On this box Hono's source built 5.2x faster (3.01 s to 0.58 s), with identical JavaScript output.

The upgrade is usually one npm install, with three real catches. Every option deprecated in TypeScript 6 is now a hard error. TypeScript 7 has no programmatic API, so typescript-eslint refuses to run. And the side-by-side workaround quietly changes which compiler next build uses. I hit all three while testing, so this guide gives the exact errors and fixes, cited to the official TypeScript 7 announcement.

What is TypeScript 7?

TypeScript 7 is a faithful port of the TypeScript compiler and language service from TypeScript to Go. It ships as the normal typescript npm package with the same tsc command. Microsoft says it "typically" yields "speedups between 8x and 12x on full builds" through native code and shared-memory multithreading. The type system and emitted JavaScript match TypeScript 6.0.

Key facts at a glance:

The announcement adds that "practically any TypeScript code that compiles cleanly with TypeScript 6.0 (with the stableTypeOrdering flag on, and without any ignoreDeprecations flag set) should compile identically in TypeScript 7.0."

Is TypeScript 7 stable and ready for production?

Yes, for compiling and type-checking. TypeScript 7.0 is a stable release, and the team tested it with companies including Bloomberg, Canva, Figma, Google, Linear, Notion, Sentry, Slack and Vercel. It is not ready for tools that need the compiler API, such as typescript-eslint and Vue, Svelte, Astro or MDX tooling, until TypeScript 7.1 ships a new API.

Quotable results from the announcement:

• Slack's CI type-checking went "from about 7.5 minutes to 1.25 minutes."

• The new language server "reduced failing language server commands by over 80%, and reduced server crashes by over 60%" compared with 6.0.

• Opening a file with an error in the VS Code codebase dropped from about 17.5 seconds to under 1.3 seconds.

The TypeScript 7.1 iteration plan lists "Stabilize API" as its first item. It scheduled the 7.1 beta for October 6 and the stable release for November 24, 2026. On October 17, 2026 npm still had no 7.1 beta, only nightly 7.1.0-dev builds under the next tag, so treat the date as a target.

How much faster is TypeScript 7? (my benchmark)

On an 8-vCPU Linux box, TypeScript 7.0.2 built Hono's source (189 files, about 26,000 lines, declarations plus JavaScript) in a median 0.58 seconds, against 3.01 seconds for TypeScript 6.0.3. That is a 5.2x speedup with 27% less peak memory. Both compilers emitted byte-identical JavaScript for all 188 output files.

The run: tsc -p tsconfig.build.json on Hono at commit 5ca49b0, three runs each, Node.js 24.22.0, measured with os.wait4 for wall time and peak RSS.

My 5.2x is below Microsoft's 8x to 12x because this project is small. Startup time weighs more on a 3-second build than on VS Code's 125-second one, so expect larger gains on larger codebases. Two things surprised me:

• More checkers didn't help here. --checkers 8 was no faster than the default of 4 and used more memory. The announcement says extra checkers help "larger codebases" at "the cost of increased memory usage."

• Even single-threaded, TypeScript 7 was 3x faster and used half the memory. On small CI runners --singleThreaded or --checkers 1 is a reasonable choice.

Does TypeScript 7 emit the same output?

The JavaScript was identical. The .d.ts files were not: 25 of 188 declaration files differed. In 16 the only change was quote style ('/' instead of "/"). In 9, TypeScript 7 wrote a reference such as toArray: typeof toArray where TypeScript 6 had inlined the full function type:

-    toArray: (children: Child) => Child[];
+    toArray: typeof toArray;

These are equivalent types, but if you publish a library, diff your generated declarations or run your API report tool before releasing a version built with TypeScript 7.

What breaks when upgrading to TypeScript 7?

Three things break most often. Every compiler option deprecated in TypeScript 6.0 is now a hard error that ignoreDeprecations no longer silences. The new defaults (types: [], rootDir: "./", strict: true) surface errors in existing configs. And any tool that imports the typescript API fails, because 7.0 ships without one.

Removed tsconfig options

I ran a deliberately old tsconfig.json through both compilers. TypeScript 6 already reports these as errors (TS5101/TS5107, "will stop functioning in TypeScript 7.0"), but you can silence them with "ignoreDeprecations": "6.0". TypeScript 7 reports TS5102/TS5108 and ignores that flag (with it set, 7.0.2 still exited 1):

tsconfig.json(3,15): error TS5108: Option 'target=ES5' has been removed. Please remove it from your configuration.
tsconfig.json(5,25): error TS5108: Option 'moduleResolution=node10' has been removed. Please remove it from your configuration.
tsconfig.json(6,5): error TS5102: Option 'baseUrl' has been removed. Please remove it from your configuration.
  Use '"paths": {"*": ["./*"]}' instead.
tsconfig.json(7,24): error TS5090: Non-relative paths are not allowed. Did you forget a leading './'?
tsconfig.json(8,5): error TS5102: Option 'downlevelIteration' has been removed. Please remove it from your configuration.
tsconfig.json(9,24): error TS5108: Option 'esModuleInterop=false' has been removed. Please remove it from your configuration.

The full list of removals and their replacements, from the announcement:

New defaults that cause errors

After removing those options, two TypeScript 6 defaults (which 7 keeps) produced new errors:

error TS5011: The common source directory of 'tsconfig.json' is './src'. The 'rootDir' setting must be
explicitly set to this or another path to adjust your output's file layout.

src/index.ts(2,19): error TS2591: Cannot find name 'process'. Do you need to install type definitions for node?
Try `npm i --save-dev @types/node` and then add 'node' to the types field in your tsconfig.

rootDir now defaults to ./, and types now defaults to [], so @types/* packages are no longer included automatically. This is the config that compiled and ran on TypeScript 7.0.2:

{
  "compilerOptions": {
    "target": "es2022",
    "module": "nodenext",
    "outDir": "dist",
    "rootDir": "./src",
    "types": ["node"]
  },
  "include": ["src"]
}

Other defaults to know: strict is true, module is esnext, noUncheckedSideEffectImports is true, and stableTypeOrdering is always on.

Language and JavaScript changes

Template literal inference now keeps Unicode code points whole. This compiles on TypeScript 7 and fails on 6, which also makes it a quick way to prove which compiler a tool is running:

// lib/which-ts.ts — compiles only on TypeScript 7
type HeadTail<S> = S extends `${infer Head}${infer Tail}` ? [Head, Tail] : never
export const check: HeadTail<'😀abc'> = ['😀', 'abc']

TypeScript 6 reports Type '"😀"' is not assignable to type '"�"', because it splits the emoji into two UTF-16 halves.

JSDoc-typed JavaScript is also stricter: @enum, @class, a bare ? type and Closure-style function(string): void are no longer recognized. The CHANGES.md file tracks the full list.

Does typescript-eslint work with TypeScript 7?

No. typescript-eslint 8.71.2 (the latest on October 17, 2026) declares typescript >=4.8.4 <6.1.0 as a peer dependency and throws at startup when it finds TypeScript 7. Because eslint-config-next uses it, npm run lint fails in every Next.js app that upgrades. The fix is to keep TypeScript 6 installed under the typescript name for the API.

This is the exact error from npx eslint . in a Next.js 16.4.2 app after npm i -D typescript@7.0.2:

typescript-eslint does not support TS 7.0.
Please see https://devblogs.microsoft.com/typescript/announcing-typescript-7-0/#running-side-by-side-with-typescript-6.0 to run typescript-eslint using the TS 6 API.
See also https://github.com/typescript-eslint/typescript-eslint/issues/10940 for tracking typescript-eslint's support for TS >=7.1

Oops! Something went wrong! :(
ESLint: 9.39.5
Error: typescript-eslint does not support TS 7.0.

Support is tracked in typescript-eslint issue #10940 and depends on the 7.1 API.

How do I run TypeScript 7 and 6 side by side?

Install TypeScript 7 under an alias such as @typescript/native, and install @typescript/typescript6 under the typescript name. Tools that import typescript (ESLint, Vue, Svelte) get the 6.0 API, while npx tsc runs the TypeScript 7 binary. This is the setup the TypeScript team recommends in its announcement.

{
  "devDependencies": {
    "@typescript/native": "npm:typescript@^7.0.2",
    "typescript": "npm:@typescript/typescript6@^6.0.2"
  }
}

What I saw after npm install:

$ npx tsc -v
Version 7.0.2
$ npx tsc6 -v
Version 6.0.3
$ npx eslint .
(no output, exit 0)

The compatibility package ships a tsc6 binary, so the two never fight over the name tsc.

The catch: next build now uses TypeScript 6

With the alias setup, next build type-checks with TypeScript 6, not 7. Next.js resolves the package named typescript, finds the compatibility package and runs its tsc6. I proved it with the emoji file above: npx tsc passed, but next build failed with the TypeScript 6 error:

  Running TypeScript ...
lib/which-ts.ts(3,42): error TS2322: Type '"😀"' is not assignable to type '"�"'.
Failed to type check.

So pick one of two setups until typescript-eslint supports 7.1:

With setup B, add npx tsc --noEmit as its own CI step to get TypeScript 7's speed for type checking. Next.js's typescript.ignoreBuildErrors would skip the slower check in next build, but only use it if that tsc step blocks the merge, because the Next.js docs call that option "very dangerous" on its own.

Does Next.js support TypeScript 7?

Yes. Since Next.js 16.3, next build can type-check with TypeScript 7, and in 16.4 it uses the project's tsc CLI by default (experimental.useTypeScriptCli: true), which works without the compiler API. Install typescript@^7 and it is used automatically. In my test the "Running TypeScript" step dropped from 1,368 ms to 231 ms.

The Next.js 16.3 release post says: "To start using TypeScript 7 for type checking during next build, just bump your project's local dependency." The TypeScript config docs add the trade-off: CLI type checking "prints the native tsc diagnostics" and "does not apply Next.js-specific code frames," and it checks every file your tsconfig.json includes, test files too.

My measurements on the Next.js 16.4.2 app from my use cache guide:

create-next-app 16.4.2 still writes "typescript": "^5" into new projects, so new apps start on 5.9 until you upgrade. The app's default tsconfig (moduleResolution: "bundler", no baseUrl) needed no changes for TypeScript 7.

TypeScript 7 upgrade checklist

To upgrade safely, get to TypeScript 6.0 with zero deprecation errors first, then install TypeScript 7, fix the types and rootDir defaults, decide how to keep the 6.0 API for ESLint and framework tooling, and add TypeScript 7 to CI. Library authors should also diff their generated .d.ts files before publishing.

1. Upgrade to TypeScript 6.0 and remove ignoreDeprecations. Fix every TS5101/TS5107 deprecation error; each becomes a hard TypeScript 7 error.

2. Replace baseUrl with ./-relative paths, and moduleResolution: "node" with "bundler" or "nodenext".

3. Install TypeScript 7: npm i -D typescript@^7 (or the alias setup if step 6 applies).

4. Add "types": ["node"] (plus jest, bun and so on) if globals like process stop resolving.

5. Set rootDir when tsconfig.json sits above src and you emit files.

6. Check API-dependent tools: typescript-eslint, eslint-config-next, Vue/Volar, Svelte, Astro, MDX, Angular templates. If you use any, keep typescript pointed at @typescript/typescript6.

7. Add npx tsc --noEmit to CI and check that it runs version 7 (npx tsc -v).

8. Tune parallelism: try --checkers 1 or --singleThreaded on small CI runners and higher --checkers on large monorepos, and measure.

9. Libraries: diff .d.ts output between 6 and 7 before publishing.

10. Editor: install the TypeScript 7 VS Code extension, and use "Disable TypeScript 7 Language Server" for Vue or Svelte projects.

Common TypeScript 7 upgrade mistakes

The most common mistakes are upgrading straight from 5.x without passing through 6.0's deprecation errors, assuming the alias setup makes next build use TypeScript 7, forgetting the new types: [] default, and trusting a passing build while lint is silently skipped. Each produces either a confusing error or a slower build than you expected.

If you deploy through Docker, rebuild without cache after changing the compiler. My Next.js Dockerfile guide shows a multi-stage build where npm run build, and with it the type check, runs in the builder stage.

Related: Prisma 8 needs TypeScript 5.9 or newer, and its client type-checked cleanly on TypeScript 7.0.2 in my test. See Prisma 8 vs Prisma 7: should you upgrade?

Related: on NestJS, TypeScript 7 isn't usable through the Nest CLI yet: nest build asks for TypeScript 6 until 7.1. Plain tsc 7 works. Details in my NestJS 12 migration guide.

Related: Oxlint's type-aware mode runs on typescript-go too. On this site's code it matched all 80 typescript-eslint findings in 1.29 s vs 20.5 s; details in Oxlint vs Biome vs ESLint.

Key takeaways

• TypeScript 7 is the native Go port of TypeScript, latest on npm since July 8, 2026 (7.0.2), with the same type system as 6.0.

• On Hono's source it built 5.2x faster than TypeScript 6 on my 8-vCPU box, with identical JavaScript output.

• Everything deprecated in TypeScript 6 is a hard error in 7: es5, node10 resolution, baseUrl, AMD/UMD modules, esModuleInterop false.

• New defaults bite: types is [] and rootDir is ./, so add "types": ["node"] and an explicit rootDir.

• typescript-eslint does not support TypeScript 7; keep @typescript/typescript6 under the typescript name until 7.1 ships an API.

• Next.js 16.3+ type-checks with TypeScript 7 automatically, but the alias setup makes next build use TypeScript 6, so run npx tsc in CI too.

• Library authors should diff .d.ts output, which can differ in quote style and typeof references.

Originally published on fabwaseem.com.

The release notes cover what changed. This guide covers what happened when I actually upgraded. I built a small but realistic NestJS 11 API with class-validator DTOs, Joi config validation, lifecycle hooks, an @Optional() dependency and e2e tests. I ran nest upgrade on it, fixed everything that broke, and benchmarked v11 against v12 for startup, throughput and memory. I also measured Zod against class-validator, Jest against Vitest, ESLint against oxlint, TypeScript 6 against TypeScript 7, and NestJS 12 on Bun. Every number below comes from those runs.

TL;DR: should I upgrade to NestJS 12?

Yes, but budget an hour rather than a minute. nest upgrade bumped every package and rewrote my config module correctly, but the build and the test suite both failed afterwards. TypeScript 6 needs three tsconfig fixes, and Jest needs --experimental-vm-modules to load the ESM-only packages. Once those are fixed, request throughput is the same as v11 (within 5%), cold start is about 90 ms slower, and you can move to Zod validation, which was 28% faster than class-validator in my test.

What's new in NestJS 12?

The NestJS v12.0.0 release groups the changes into five areas:

• ESM packages. Every core @nestjs/* package now ships as ESM. A CommonJS app loads them through Node's require(esm), so you don't have to convert your own code.
• Standard Schema. @Body(), @Query(), @Param() and @RawBody() take a schema option. The new StandardSchemaValidationPipe and StandardSchemaSerializerInterceptor validate requests and responses with any Standard Schema library. @nestjs/config now accepts a Zod schema as validationSchema too.
• A rebuilt CLI. It adds nest upgrade (with --dry-run), nest deploy, Rspack as the default bundler for monorepos, and bun as a package manager. nest new scaffolds ESM, Vitest and oxlint by default.
• Native observability. @nestjs/observe hooks into Nest's own request lifecycle instead of patching the HTTP server like a generic APM agent.
• Smaller additions. These include routeConflictPolicy and routeResolutionStrategy for route shadowing, machine-readable errorCode on HTTP exceptions, structured logging params, and graceful shutdown in the Express adapter.

The breaking changes that matter most for typical REST APIs are listed in the official migration guide: ESM packages, lifecycle hook ordering by component hierarchy, @Optional() inheritance, @nestjs/config moving to Standard Schema (Joi must be v18+), NATS v3, graphql-ws-only subscriptions, and the removal of the legacy Terminus health indicator API.

How I tested the NestJS 12 upgrade

• App: a NestJS 11.2.7 project scaffolded with CLI 11.0.24 (Express adapter), plus a users module. It has a class-validator DTO, @nestjs/config 4 with a Joi 17 schema and custom validationOptions, onModuleInit and onApplicationBootstrap hooks, a repository subclass that inherits an @Optional() constructor dependency, a GET /users/me route declared after GET /users/:id, and Jest e2e tests with supertest.
• Upgrade: CLI 12.0.8 nest upgrade, which installed NestJS 12.1.3, @nestjs/config 12.0.1 and Joi 18.3.0.
• Runtime: Node.js 24.22.0 on an 8 vCPU Xeon VM. Node.js 26.12.0 and Bun 1.4.3 were used for the cross-runtime checks.
• Benchmarks: hyperfine for startup and tooling times (20 runs for startup, 5 to 8 for tools). For HTTP I used oha with 64 connections, taking the median of three 10-second runs after a warm-up, with the server pinned to 2 cores and oha on the other 6.

"Cold start" means the wall time to start the process, boot Nest, bind the port and close it again. Inside the process, NestFactory.create() plus listen() took 70 ms on v11 and 84 to 93 ms on v12. The rest is Node starting and loading modules.

How do I upgrade to NestJS 12?

The upgrade has five steps. The first trips almost everyone coming from a v11 project.

Step 1: upgrade the CLI (and TypeScript with it)

The guide says to upgrade the CLI first, both globally and in your project. On my v11 project the local install failed:

$ npm i -D @nestjs/cli@latest @nestjs/schematics@latest
npm error ERESOLVE unable to resolve dependency tree
npm error Found: typescript@5.9.3
npm error peer typescript@">=6.0.0" from @nestjs/schematics@12.0.7

@nestjs/schematics 12 needs TypeScript 6 or newer as a peer, and a v11 project has 5.x. Skipping the local install doesn't help either. The global v12 binary hands off to the project's local v11 CLI, which answers Invalid command: upgrade. The fix is to bump TypeScript in the same command:

npm i -D @nestjs/cli@latest @nestjs/schematics@latest typescript@^6

Use TypeScript 6, not 7. typescript@latest is 7.0.2 now, and the Nest CLI can't build with it yet (more on that below). The schematics also need Node.js 22.22.3+, 24.15+ or 26+, which is a higher floor than running the app.

Step 2: run nest upgrade --dry-run

npx nest upgrade --dry-run

On my app, the dry run printed this report and changed nothing:

Changes applied:
  ✔ Updated NestJS packages: @nestjs/common@^12.0.0, @nestjs/core@^12.0.0, @nestjs/testing@^12.0.0,
    @nestjs/platform-express@^12.0.0, @nestjs/cli@^12.0.0, @nestjs/schematics@^12.0.0, @nestjs/config@^12.0.0
  ✔ Updated "ts-jest" from "^29.2.5" to "^29.4.0" (first release supporting Jest 30)
  ✔ /src/app.module.ts: moved "validationOptions" settings under "validationOptions.libraryOptions"
  ✔ Updated "joi" from "^17.13.8" to "^18.0.0" (Joi v18 implements Standard Schema, required by @nestjs/config v12)
Notes:
  ℹ Lifecycle hooks are now invoked by component hierarchy level (found in /src/users/users.service.ts)
Please review:
  ⚠ Jest: the NestJS 12 packages are ESM-only and Jest can only require() ESM modules on Node.js v24.9+
Action required:
  ➜ tsconfig.build.json does not set "rootDir". TypeScript 6 requires it explicitly (error TS5011)

It's a good report. It found the Joi schema, the config option that needs to move, and the file that uses lifecycle hooks. Add --no-observe (or --observe) to skip the prompt about setting up @nestjs/observe.

Step 3: run nest upgrade

npx nest upgrade --no-observe

This took 6 seconds including the install. It changed 3 files: package.json, package-lock.json and one line in app.module.ts:

-      validationOptions: { allowUnknown: true, abortEarly: true },
+      validationOptions: { libraryOptions: { allowUnknown: true, abortEarly: true } },

It doesn't touch your module format, so a CommonJS app stays CommonJS. It doesn't switch you to Vitest or oxlint either.

Step 4: fix the TypeScript 6 errors

After the upgrade, nest build failed with five errors:

error TS5011: The common source directory of 'tsconfig.build.json' is './src'. The 'rootDir' setting must be explicitly set
error TS5101: Option 'baseUrl' is deprecated and will stop functioning in TypeScript 7.0.
src/users/create-user.dto.ts:3:14 - error TS2564: Property 'email' has no initializer and is not definitely assigned in the constructor.

These come from TypeScript 6, not from NestJS. TypeScript 6 turns strict on by default, which brings in strictPropertyInitialization, and that flags every class-validator DTO property. It also deprecates baseUrl and makes rootDir explicit. The v12 project template sets these options:

// tsconfig.json
{
  "compilerOptions": {
    "rootDir": ".",
    "strictPropertyInitialization": false
    // and delete "baseUrl": "./" unless you use path aliases
  }
}

// tsconfig.build.json
{
  "extends": "./tsconfig.json",
  "compilerOptions": { "rootDir": "./src" },
  "include": ["src"],
  "exclude": ["node_modules", "test", "dist", "**/*spec.ts"]
}

The upgrade report only mentioned tsconfig.build.json. My e2e tests use the base tsconfig.json through ts-jest, and they failed with the same TS5011 error until I added "rootDir": "." there too. If you use path aliases, replace baseUrl with paths entries relative to the tsconfig, or set "ignoreDeprecations": "6.0" for now. My TypeScript 7 upgrade guide covers the rest of the 6.0 and 7.0 changes.

Step 5: make Jest load the ESM packages

Once it compiled, every Jest suite failed:

Must use import to load ES Module: node_modules/@nestjs/testing/index.js
  - Use Node v24.9+ where Jest supports require(esm) natively

I was already on Node 24.22, and Node 26.12 failed the same way. Jest 30.5.2 only takes its ESM path when vm.SyntheticModule exists, and Node only exposes it behind a flag. Setting that flag fixed all six tests:

"test:e2e": "NODE_OPTIONS=--experimental-vm-modules jest --config ./test/jest-e2e.json"

(Use cross-env if you need Windows support.) The alternative is to move to Vitest, which is what new v12 projects use. More on that below.

Which NestJS 12 breaking changes actually bit?

@Optional() inheritance depends on the exact v12 version

The migration guide says a subclass without its own constructor no longer inherits its parent's @Optional() markers, and that Nest now throws UnknownDependenciesException where v11 injected undefined. I tested this directly with @nestjs/testing:

On v11, both subclasses silently got undefined for the missing dependency. 12.1.0 added a lookup that finds the class which actually declares the constructor, which brought the inherited markers back for subclasses without a constructor. A subclass that redeclares its constructor without @Optional() throws on every v12 release, and on early 12.0.x releases the constructor-less subclass throws too. Declare the marker again on the subclass:

@Injectable()
class UsersRepo extends BaseRepo {
  constructor(@Optional() @Inject(AUDIT) audit?: AuditLog) {
    super(audit);
  }
}

Lifecycle hook order

nest upgrade flagged my hooks file. The order of onModuleInit and onApplicationBootstrap across my repository and service was identical on v11 and v12. The change only affects providers at different levels of the component hierarchy. If your teardown or startup code depends on another provider's hook having run, add a test that checks the order.

Route shadowing is still opt-in

GET /users/me declared after GET /users/:id still returned the :id handler's 400 error on v12, the same as v11. The new options are off by default. Turn them on to get warnings and literal-before-parameter ordering:

const app = await NestFactory.create(AppModule, {
  routeConflictPolicy: { duplicate: 'error', shadow: 'warn' },
  routeResolutionStrategy: 'specificity',
});

With both set, /users/me returned {"me":true}, and Nest logged a WARN line for each literal and parameter route pair that overlaps. I recommend turning both on in new apps.

How do I use Zod for validation in NestJS 12?

Add a schema to the parameter decorator and register StandardSchemaValidationPipe. No custom pipe or extra package is needed:

import { z } from 'zod';

export const createUserSchema = z.object({
  email: z.email(),
  name: z.string().min(2),
  age: z.number().int().min(13).max(120),
});
export type CreateUser = z.infer<typeof createUserSchema>;

@Controller('zusers')
export class ZUsersController {
  constructor(private users: UsersService) {}

  @Post()
  create(@Body({ schema: createUserSchema }) body: CreateUser) {
    return this.users.create(body);
  }

  @Get(':id')
  find(@Param('id', { schema: z.coerce.number().int().positive() }) id: number) {
    return this.users.find(id);
  }
}

// main.ts: both pipes can run side by side during a migration
app.useGlobalPipes(
  new ValidationPipe({ whitelist: true, transform: true }),
  new StandardSchemaValidationPipe(),
);

With both pipes registered globally, class-validator DTO routes and Zod routes work in the same app. Each pipe only acts on parameters it recognises, so you can migrate one controller at a time. Invalid input returns the usual 400 response, with messages prefixed by field path:

{"message":["email: Invalid email address","name: Too small: expected string to have >=2 characters","age: Too small: expected number to be >=13"],"error":"Bad Request","statusCode":400}

Unknown keys are stripped by default, as with whitelist: true, because z.object drops them.

On the same app, server and load, POST /zusers with Zod 4.6.7 handled 20,804 req/s, and POST /users with class-validator handled 16,237 req/s. That makes Zod 28% faster, with p99 latency of 6.2 ms against 7.3 ms. Zod also gives you a single source for the TypeScript type, the validation and the OpenAPI schema. On v12 I'd use it for new endpoints. class-validator stays fully supported, so there's no need to rewrite working DTOs.

Machine-readable error codes come with v12 too. throw new BadRequestException('Password is too weak', { errorCode: 'WEAK_PASSWORD' }) returned {"message":"Password is too weak","error":"Bad Request","statusCode":400,"errorCode":"WEAK_PASSWORD"}, so clients can branch on a stable code instead of parsing the message.

Is NestJS 12 slower than NestJS 11?

Request handling is not slower. Startup is.

Throughput moved by 5% or less in either direction, which is within the run-to-run variation. Cold start grew by about 90 ms (30%) in a CommonJS app, and most of that is the framework's own import cost: 50 ms more just to load the three core packages. The CJS app's require(esm) path is not the main cause, because the same app converted to ESM was only 23 ms faster. Adding Zod and a second controller added another 44 ms.

For a long-running API, 90 ms at boot doesn't matter. For serverless and scale-to-zero it does, and two cheap fixes help:

• Enable Node's compile cache. With NODE_COMPILE_CACHE=/tmp/node-cache, v12 started in 338 ms instead of 386 ms, and v11 in 257 ms.
• Run on Bun. The same v12 ESM build started in 264 ms on Bun 1.4.3, faster than v11 on Node.

Node 26.12 started the v12 app in 355 ms, close to Node 24 with the compile cache.

Should I convert my NestJS app to ESM?

You don't have to: nest upgrade leaves CommonJS apps alone, and they run fine. Converting gave me 23 ms of startup and nothing else measurable. Do it when a dependency you need is ESM-only and uses top-level await (which require(esm) can't load), or when you want Vitest without extra configuration.

The conversion took me ten minutes on a small app:

1. Add "type": "module" to package.json. Projects generated with @nestjs/schematics 11.0.6+ already use "module": "nodenext", so tsconfig needs no other change.
2. Add .js to every relative import ('./app.module.js'). A one-line regex changed 9 files.
3. Replace __dirname with import.meta.dirname and call await bootstrap().
4. Fix namespace imports of CommonJS packages. My build passed, but startup crashed with TypeError: Joi.string is not a function until I changed import * as Joi from 'joi' to import Joi from 'joi'. TypeScript doesn't catch this one, so start the app after converting.

Jest vs Vitest and ESLint vs oxlint in NestJS 12

A new v12 project (nest new, which defaulted to ESM when run without a TTY) gets this package.json: "type": "module", vitest 4 with vite 8, oxlint plus oxlint-tsgolint for type-aware rules, and TypeScript 6. There's no SWC plugin. I copied the template's Vitest config into my converted app, and all six e2e tests passed unchanged, including DI that depends on decorator metadata.

Jest got 33% slower on v12 because of the ESM loading path. Vitest was 40% faster than that, and 20% faster than Jest on v11. oxlint was 5.5 times faster than ESLint on this small codebase, and the gap grows with more files. It's not a drop-in replacement, though. ESLint reported 61 problems (35 of them Prettier formatting and 25 no-unsafe-* rules), while oxlint reported one warning that ESLint didn't (no-misused-spread). Keep Prettier as a separate step if you move to oxlint.

Does NestJS 12 support TypeScript 7?

You can't build NestJS 12 with TypeScript 7 through the Nest CLI yet. With typescript@7.0.2 installed, nest build stops with:

Error  The installed TypeScript version (7.0.2) does not expose the programmatic compiler API that the Nest CLI requires.
TypeScript 7.0 ships the "tsc" executable only; the compiler API is expected to return in 7.1.
Please install TypeScript 6 (e.g. "npm i -D typescript@^6") until then.

nest build -b swc stops with the same error. Plain tsc -p tsconfig.build.json from TypeScript 7 compiled the app correctly, and the output booted normally. So if you want the 3x faster type-check now, run tsc 7 for type-checking and builds in CI, and keep TypeScript 6 for nest start --watch until 7.1.

Can I run NestJS 12 on Bun?

Yes. The compiled ESM app ran unchanged with bun dist/main.js, including class-validator, Joi config and Express. Under the same load test, Bun 1.4.3 beat Node 24.22:

The v12 CLI also accepts bun as a package manager. For the broader trade-offs (memory over time, APM agents, native addons), see my Bun vs Node benchmark.

NestJS 12 upgrade checklist

1. Make sure you're on Node.js 22.22.3+, 24.15+ or 26 for the CLI. The app itself needs only 20.19+ or 22.12+.
2. Run npm i -D @nestjs/cli@latest @nestjs/schematics@latest typescript@^6.
3. Run npx nest upgrade --dry-run and read the report.
4. Run npx nest upgrade --no-observe and review the diff.
5. tsconfig: add rootDir to both files, remove baseUrl, and set strictPropertyInitialization: false if you use class DTOs.
6. Tests: add NODE_OPTIONS=--experimental-vm-modules for Jest, or move to Vitest.
7. Search for subclasses that redeclare a constructor and add @Optional() where needed.
8. Run the app and hit every route, especially if you use NATS, GraphQL subscriptions, Terminus or custom pipes.
9. Optionally turn on routeConflictPolicy and routeResolutionStrategy: 'specificity', write new endpoints with Zod, and set NODE_COMPILE_CACHE in production.

On AWS Lambda's Node 20, 22 and 24 runtimes, require(esm) is off by default. Set NODE_OPTIONS=--experimental-require-module there for a CommonJS app.

Common mistakes when upgrading to NestJS 12

• Installing typescript@latest. That's 7.0, and the Nest CLI refuses to build with it. Pin ^6 until TypeScript 7.1.
• Upgrading only the global CLI. The global binary hands off to the project's local v11 CLI, which doesn't know upgrade.
• Trusting the "Node 24.9+" Jest note. On Node 24.22 and 26.12, Jest 30 still needed --experimental-vm-modules to load @nestjs/testing.
• Fixing only tsconfig.build.json. Tests compile with tsconfig.json, which needs rootDir too.
• Converting to ESM and keeping import * as X for CommonJS packages. It compiles and then crashes at startup.
• Missing redeclared @Optional() markers. A subclass that redeclares its constructor now throws UnknownDependenciesException instead of injecting undefined, and 12.0.0 to 12.0.4 also throw for subclasses without a constructor. Use 12.1.x and re-mark optional parameters.
• Rewriting every DTO to Zod at once. Both pipes run side by side. Migrate one controller at a time.

Related: if you're choosing a database layer for your Nest API, read Drizzle vs Prisma in 2026. If you're exposing the API to AI agents, read MCP vs API.

Related: NestJS on the Fastify adapter was 1.45 to 1.66× faster than on Express in my Hono vs Express vs Fastify benchmark. And since nest new now scaffolds Oxlint, see Oxlint vs Biome vs ESLint: Oxlint ran this site's ESLint rules 81× faster.

Key takeaways

• NestJS 12 shipped on August 27, 2026, with ESM-only packages, Standard Schema validation, a rebuilt CLI and @nestjs/observe. 12.1.3 is current.
• nest upgrade --dry-run gives an accurate report, but upgrade the local CLI together with typescript@^6 or npm fails with ERESOLVE.
• After the upgrade, fix TypeScript 6's rootDir (in both tsconfigs), baseUrl and strict DTO errors, and give Jest --experimental-vm-modules.
• Throughput stayed within 5% of v11. Cold start grew by about 90 ms, and NODE_COMPILE_CACHE or Bun wins most of it back.
• Zod through @Body({ schema }) was 28% faster than class-validator, 20,804 vs 16,237 req/s, and both pipes can run in one app.
• Vitest ran the same e2e suite 40% faster than Jest on v12, and oxlint linted 5.5 times faster than ESLint.
• TypeScript 7 can type-check and build a Nest 12 app with plain tsc, but nest build needs TypeScript 6 until 7.1.

That is great for scaling and painful if your server kept state in the session. In this guide I build a stateless MCP server with the official TypeScript SDK v2 (@modelcontextprotocol/server 2.3.1). I run two copies behind different ports and send real requests to both with curl and the SDK client. You will see exactly what changed on the wire, how to replace session state with handles and requestState, and the errors you get when something is wrong. Every rule is cited to the spec.

What is a stateless MCP server?

A stateless MCP server handles every JSON-RPC request on its own, with no handshake and no session. Under the 2026-07-28 spec each request carries its protocol version, client capabilities and client info in _meta, plus MCP-Protocol-Version, Mcp-Method and Mcp-Name HTTP headers. Any instance behind a plain load balancer can answer it, with no sticky sessions or session store.

The official 2026-07-28 release post says it directly: "MCP is transforming from a bidirectional stateful protocol into a request/response stateless protocol," and it calls this "one of the most highly-requested features" from developers. It also says "any request can land on any instance behind a plain round-robin load balancer."

What changed in the MCP 2026-07-28 spec?

The 2026-07-28 revision removed sessions, the initialize handshake, the HTTP GET stream, SSE resumability and server-to-client requests. It added server/discover, per-request _meta, required routing headers, a resultType on every result, cache fields on list results, subscriptions/listen for change notifications, and Multi Round-Trip Requests for elicitation and sampling.

The full list from the official changelog, grouped by what you have to do:

The spec also deprecated Roots, Sampling, Logging, the old HTTP+SSE transport and Dynamic Client Registration. Under the new feature lifecycle policy deprecated features get a minimum twelve-month window before removal.

Should an MCP server be stateless?

Yes. Since 2026-07-28 the protocol itself is stateless, so every modern MCP server is stateless at the protocol level. Your application can still keep state, but in your own database, addressed by explicit handles that tools return and the model passes back. That works better for models too, because they can see and reuse the handle.

The release post explains why: "If your server needs to carry state across calls, mint an explicit handle from a tool and have the model pass it back as an argument. We found this works better than session state hidden in the transport - the model can see the handle and thread it between tools."

AWS's architecture team reached the same conclusion for their platform: the sticky sessions and shared session stores a session-based server needed "are no longer necessary" (AWS Architecture Blog).

How do I build a stateless MCP server in TypeScript?

Install @modelcontextprotocol/server v2, write a factory that builds a fresh McpServer and registers your tools, and pass it to createMcpHandler. The handler serves 2026-07-28 requests and, by default, 2025-era requests too, all per request. On Node, wrap it with toNodeHandler from @modelcontextprotocol/node; on web runtimes, call handler.fetch(request).

Step 1: Install

Tested on Node.js 24.22.0:

npm i @modelcontextprotocol/server@2.3.1 @modelcontextprotocol/node@2.1.1 @modelcontextprotocol/client@2.3.1 zod@^4

The SDK depends on Zod 4 and you import it as zod/v4.

Step 2: Keep state in a store, not a session

The demo is a tiny cart service. The cart lives in a shared store keyed by a server-minted handle. I used a JSON file so two processes share it; in production this is Postgres or Redis.

// store.ts
import { readFileSync, writeFileSync, existsSync } from 'node:fs'

type Cart = { owner: string; items: { sku: string; qty: number }[]; createdAt: string }
const FILE = new URL('./data/carts.json', import.meta.url)

function load(): Record<string, Cart> {
  return existsSync(FILE) ? JSON.parse(readFileSync(FILE, 'utf8')) : {}
}
function save(all: Record<string, Cart>) {
  writeFileSync(FILE, JSON.stringify(all, null, 2))
}

export const carts = {
  create(owner: string): string {
    const all = load()
    const id = `cart_${crypto.randomUUID()}`
    all[id] = { owner, items: [], createdAt: new Date().toISOString() }
    save(all)
    return id
  },
  get(id: string, owner: string): Cart | undefined {
    const cart = load()[id]
    return cart && cart.owner === owner ? cart : undefined // a handle is not a credential
  },
  update(id: string, cart: Cart) {
    const all = load()
    all[id] = cart
    save(all)
  },
}

Note the owner check. A handle such as cart_... travels through the model, so treat it as an identifier, never as proof of access. Always check it against the authenticated user.

Step 3: Write the server

// server.ts
import { createServer } from 'node:http'
import {
  McpServer,
  createMcpHandler,
  createRequestStateCodec,
  inputRequired,
  acceptedContent,
} from '@modelcontextprotocol/server'
import type { CallToolResult, InputRequiredResult } from '@modelcontextprotocol/server'
import { toNodeHandler } from '@modelcontextprotocol/node'
import * as z from 'zod/v4'
import { carts } from './store.ts'

const PORT = Number(process.env.PORT ?? 3101)
const INSTANCE = process.env.INSTANCE ?? `instance-${PORT}`

// Shared by every instance, so state minted on one instance verifies on another.
const stateCodec = createRequestStateCodec<{ step: 'confirm-clear'; cartId: string }>({
  key: new TextEncoder().encode(process.env.MCP_STATE_KEY ?? 'dev-only-key-change-me-32-bytes!!'),
  ttlSeconds: 300,
})

function buildServer(): McpServer {
  const server = new McpServer(
    { name: 'shop-mcp', version: '1.0.0' },
    {
      requestState: { verify: stateCodec.verify },
      cacheHints: { 'tools/list': { ttlMs: 60_000, cacheScope: 'public' } },
    },
  )
  // Demo principal. In a real server this comes from the verified bearer token (authInfo).
  const owner = 'demo-user'

  server.registerTool(
    'create_cart',
    { description: 'Create a shopping cart and return its cart_id handle. Pass cart_id to the other cart tools.' },
    async (): Promise<CallToolResult> => {
      const cartId = carts.create(owner)
      return {
        content: [{ type: 'text', text: `Created ${cartId} on ${INSTANCE}` }],
        structuredContent: { cart_id: cartId },
      }
    },
  )

  server.registerTool(
    'add_item',
    {
      description: 'Add a product to a cart created with create_cart',
      inputSchema: z.object({ cart_id: z.string(), sku: z.string(), qty: z.number().int().min(1) }),
    },
    async ({ cart_id, sku, qty }): Promise<CallToolResult> => {
      const cart = carts.get(cart_id, owner)
      if (!cart) return { content: [{ type: 'text', text: `Unknown cart_id ${cart_id}` }], isError: true }
      cart.items.push({ sku, qty })
      carts.update(cart_id, cart)
      return { content: [{ type: 'text', text: `${cart.items.length} line(s) in ${cart_id}, served by ${INSTANCE}` }] }
    },
  )

  const confirmSchema = z.object({ confirm: z.boolean() })
  server.registerTool(
    'clear_cart',
    {
      description: 'Remove every item from a cart after the user confirms',
      inputSchema: z.object({ cart_id: z.string() }),
    },
    async ({ cart_id }, ctx): Promise<CallToolResult | InputRequiredResult> => {
      const cart = carts.get(cart_id, owner)
      if (!cart) return { content: [{ type: 'text', text: `Unknown cart_id ${cart_id}` }], isError: true }
      const state = ctx.mcpReq.requestState<{ step: 'confirm-clear'; cartId: string }>()
      const answer = acceptedContent(ctx.mcpReq.inputResponses, 'confirm', confirmSchema)
      if (state?.cartId !== cart_id || answer?.confirm !== true) {
        // Multi Round-Trip Request: ask the client, then expect a retry of this same call.
        return inputRequired({
          inputRequests: {
            confirm: inputRequired.elicit({
              message: `Remove all ${cart.items.length} line(s) from ${cart_id}?`,
              requestedSchema: confirmSchema,
            }),
          },
          requestState: await stateCodec.mint({ step: 'confirm-clear', cartId: cart_id }),
        })
      }
      cart.items = []
      carts.update(cart_id, cart)
      return { content: [{ type: 'text', text: `Cleared ${cart_id} on ${INSTANCE}` }] }
    },
  )
  return server
}

// Fresh McpServer per request; no sessions to store or route.
const handler = createMcpHandler(buildServer, { responseMode: 'json' })

createServer(toNodeHandler(handler)).listen(PORT, '127.0.0.1', () => {
  console.log(`${INSTANCE} listening on http://127.0.0.1:${PORT}/mcp`)
})

The file type-checks with TypeScript 7.0.2 in strict mode and runs directly with node server.ts on Node 24, which strips types natively. I started two instances to stand in for two containers behind a load balancer:

PORT=3101 INSTANCE=A node server.ts &
PORT=3102 INSTANCE=B node server.ts &

On a Next.js App Router project the same handler goes in app/api/mcp/route.ts as export const POST = (req: Request) => handler.fetch(req). My guide on building an MCP server in Next.js covers that route with Origin checks and bearer auth. To teach coding agents when and how to call your tools, pair the server with an Agent Skill.

What does a stateless MCP request look like?

A stateless MCP request is a single HTTP POST with three headers, MCP-Protocol-Version, Mcp-Method and (for tool calls) Mcp-Name, and a JSON-RPC body whose params._meta repeats the protocol version and declares the client's capabilities. There is no earlier handshake: the first request a client sends can be the tool call itself.

This is the exact request my curl helper sends for create_cart:

POST /mcp HTTP/1.1
Content-Type: application/json
Accept: application/json, text/event-stream
MCP-Protocol-Version: 2026-07-28
Mcp-Method: tools/call
Mcp-Name: create_cart

{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"create_cart","arguments":{},
 "_meta":{"io.modelcontextprotocol/protocolVersion":"2026-07-28",
          "io.modelcontextprotocol/clientInfo":{"name":"curl","version":"8"},
          "io.modelcontextprotocol/clientCapabilities":{}}}}

server/discover

The SDK implements server/discover for you. Real response from instance A:

{"result":{"supportedVersions":["2026-07-28"],"capabilities":{"tools":{"listChanged":true}},
 "resultType":"complete","ttlMs":0,"cacheScope":"private",
 "_meta":{"io.modelcontextprotocol/serverInfo":{"name":"shop-mcp","version":"1.0.0"}}},
 "jsonrpc":"2.0","id":1}

tools/list with cache hints

Because I set cacheHints, the list result tells clients they may cache it for 60 seconds:

{"resultType":"complete","ttlMs":60000,"cacheScope":"public","tools":["create_cart","add_item","clear_cart"]}

Without a hint the SDK sends ttlMs: 0 and cacheScope: "private", the most conservative policy. If your tool list differs per user, keep cacheScope at "private". The spec says "public" allows shared intermediaries to cache the response.

One cart, two instances

I created a cart on instance A, then added items through B and then A. Every call landed on a different process and nothing broke, because the state is in the store, not the connection:

create_cart  on A  → Created cart_d892bc05-… on A
add_item     on B  → 1 line(s) in cart_d892bc05-…, served by B
add_item     on A  → 2 line(s) in cart_d892bc05-…, served by A

How do I replace session state when migrating?

Replace session state in three ways. Data that outlives one call goes in your database under a handle a tool returns, such as cart_id. Data needed only between the rounds of one interaction goes in requestState, signed with HMAC. Per-user data comes from the auth token on each request, never from a session ID.

The TypeScript SDK's 2026-07-28 migration notes state that requestState "round-trips through the client and is therefore untrusted input" and tell you to integrity-protect it. createRequestStateCodec does that with HMAC-SHA256, and the notes add that it is "signed, not encrypted", so never put secrets in it.

How do elicitation and confirmations work without sessions?

Without sessions the server can no longer push elicitation/create in the middle of a call. Instead it returns a result with resultType: "input_required", the questions in inputRequests, and an optional requestState. The client asks the user, then retries the same call with inputResponses and the requestState echoed back. This pattern is called Multi Round-Trip Requests (MRTR).

Round 1, sent to instance B with clientCapabilities: {"elicitation":{"form":{}}}:

{"result":{"resultType":"input_required",
 "inputRequests":{"confirm":{"method":"elicitation/create","params":{
   "message":"Remove all 2 line(s) from cart_d892bc05-…?",
   "requestedSchema":{"type":"object","properties":{"confirm":{"type":"boolean"}},"required":["confirm"]},
   "mode":"form"}}},
 "requestState":"v1.eyJwIjp7InN0ZXAiOiJjb25maXJtLWNsZWFyIiwi…"}}

Round 2, the retry, sent to instance A with "inputResponses":{"confirm":{"action":"accept","content":{"confirm":true}}} and the same requestState:

{"result":{"content":[{"type":"text","text":"Cleared cart_d892bc05-… on A"}],"resultType":"complete"}}

Instance A verified state minted by instance B because both share MCP_STATE_KEY. When I changed one character of requestState, the server rejected it:

{"jsonrpc":"2.0","id":1,"error":{"code":-32602,"message":"Invalid or expired requestState","data":{"reason":"invalid_request_state"}}}

With the SDK client you don't write the retry yourself. It answers input_required through your elicitation/create handler and retries automatically:

// client.ts
import { Client, StreamableHTTPClientTransport } from '@modelcontextprotocol/client'

const client = new Client(
  { name: 'shop-client', version: '1.0.0' },
  {
    versionNegotiation: { mode: 'auto' }, // the v2 client defaults to the 2025 handshake
    capabilities: { elicitation: { form: {} } },
  },
)

client.setRequestHandler('elicitation/create', async (request) => {
  console.log('server asks:', request.params.message)
  return { action: 'accept', content: { confirm: true } }
})

await client.connect(new StreamableHTTPClientTransport(new URL('http://127.0.0.1:3101/mcp')))
console.log('era:', client.getProtocolEra())

Output:

era: modern
server asks: Remove all 1 line(s) from cart_3ab350d7-…?
[ { type: 'text', text: 'Cleared cart_3ab350d7-… on A' } ]

What errors does a stateless MCP server return?

A 2026-07-28 server returns HTTP 400 with JSON-RPC code -32020 when headers are missing or disagree with the body, -32022 for an unsupported protocol version, and -32021 when a tool needs a client capability the request did not declare. It returns 404 with -32601 for unknown or removed methods such as ping, and 405 for GET.

All of these are real responses from my server:

The header check exists for security. The Streamable HTTP spec says servers must reject mismatches because otherwise "a load balancer routing on the header value while the MCP server executes based on the body value" could be tricked.

Will old MCP clients still work with my stateless server?

Mostly yes, with one important exception. createMcpHandler defaults to legacy: 'stateless', which answers 2025-era initialize and later requests per request without issuing a session ID. Ordinary tools work. Tools that need user input fail for old clients, because a per-request legacy connection cannot receive server-to-client requests.

The legacy initialize returned 200, content-type: text/event-stream and no Mcp-Session-Id header:

data: {"result":{"protocolVersion":"2025-11-25","capabilities":{"tools":{"listChanged":true}},"serverInfo":{"name":"shop-mcp","version":"1.0.0"}},"jsonrpc":"2.0","id":1}

Then I ran the same client with versionNegotiation: { mode: 'legacy' }. create_cart and add_item worked, but clear_cart returned:

Cannot request input 'confirm' (elicitation/create): the client on this 2025-era connection did not declare
the required capability (no client capabilities are available on this connection — per-request legacy
serving cannot receive server-to-client requests)

So if old clients need confirmations, keep your existing sessionful handler for them. The SDK's migration notes show how: put isLegacyRequest(request) in front of a createMcpHandler(factory, { legacy: 'reject' }) entry and route legacy traffic to your old handler.

Two client-side facts worth knowing:

• The v2 Client defaults to the 2025 handshake. You must opt in with versionNegotiation: { mode: 'auto' } (probe with server/discover, fall back) or { pin: '2026-07-28' }.
• The versioning page has a full compatibility matrix. A legacy client talking to a modern-only server fails, because, in the spec's words, "Legacy clients have no fall-forward mechanism."

Migration checklist: stateful to stateless MCP

To migrate, upgrade to an SDK that implements 2026-07-28, switch to a per-request handler, move session data into handles, requestState or auth, and rewrite pushed elicitation as input_required. Then remove session infrastructure, set cache hints, and test with both modern and legacy clients before you drop sticky routing.

1. Upgrade the SDK. TypeScript: move from @modelcontextprotocol/sdk 1.x to @modelcontextprotocol/server 2.x (createMcpHandler). Python, Go and C# also have updated Tier 1 SDKs, per the release post.
2. Find every use of the session ID (extra.sessionId, ctx.sessionId, maps keyed by Mcp-Session-Id) and move that data to a handle, requestState or the auth principal.
3. Return handles from tools (cart_id, job_id, doc_id) and name them in tool descriptions so the model passes them back.
4. Authorize every handle against the authenticated user.
5. Rewrite elicitInput() / requestSampling() as return inputRequired({...}) and read ctx.mcpReq.inputResponses on re-entry.
6. Seal requestState with createRequestStateCodec and share the key across instances.
7. Make tools safe to retry. Streams are no longer resumable, so a dropped connection means the client re-issues the request.
8. Set cacheHints for tools/list and resources, private if results vary per user.
9. Delete ping handlers and per-session log level code; log to stderr or OpenTelemetry instead.
10. Change resource-not-found errors from -32002 to -32602.
11. Test both eras: a modern client (mode: 'auto') and a legacy client, including tools that ask for input.
12. Only then remove sticky sessions, the session store and the GET route.

Common stateless MCP mistakes

The most common mistakes are trusting a handle as authorization, putting secrets in requestState, forgetting the routing headers in a custom client, marking per-user lists as public, and assuming every old client can still answer elicitation. Each of these either fails with a 400 you can see or leaks data silently, so test them on purpose.

Related: stateless or not, every MCP call costs more than a REST call. I measured how much, in latency, throughput and tokens, in MCP vs API: when you need which.

Key takeaways

• The MCP 2026-07-28 spec made the protocol stateless: no initialize, no Mcp-Session-Id, every request self-describing.
• Servers must implement server/discover and reject header/body mismatches with -32020.
• Keep application state in your database behind handles that tools return; authorize every handle.
• Elicitation is now Multi Round-Trip Requests: return input_required, the client retries with inputResponses and requestState.
• Sign requestState with createRequestStateCodec and share the key so any instance can verify it.
• createMcpHandler also serves 2025-era clients per request, but they cannot answer elicitation on that path.
• The v2 TypeScript client still defaults to the 2025 handshake; opt in with versionNegotiation: { mode: 'auto' }.

This guide covers the real advisories, with links and affected versions. I also reproduced the TypeScript SDK flaw on my own machine to show what leaks, which versions stop it, and the one setting most people will miss after upgrading. I then walk through how this site's own MCP server at https://www.fabwaseem.com/api/mcp handles OAuth, scopes and audit logging, with measurements. Nothing here is hypothetical: every vulnerability has a public advisory, and every number comes from a run.

TL;DR: is MCP secure?

The protocol's authorization spec is sound. The problems are in implementations: clients that trust whatever a server says, and servers that skip audience checks or expose too much. If you build an MCP client, upgrade and pass expectedIssuer. In my lab, @modelcontextprotocol/sdk 1.31.0 and 1.32.1 still sent a client secret to the attacker's authorization server when ClientCredentialsProvider was created without it. If you run an MCP server, validate the token's audience, scope every tool and deny unmapped tools by default, and keep an audit log.

What happened to MCP security in October 2026?

Four separate issues landed in about a month, and three share a root cause: one side trusted what the other side said about who it was.

• Both official SDKs: issuer confusion. In MCP's OAuth flow, the client asks the MCP server where to sign in, using protected resource metadata. Affected SDK versions didn't tie stored credentials to the authorization server they came from. A malicious or compromised MCP server could name its own authorization server and receive the client's client_secret, signed client assertion or saved refresh_token, with no user interaction. The TypeScript advisory was published on September 30, 2026, and the Python one on September 28.
• Rust SDK: missing resource check. rmcp before 2.0.0 didn't check that the resource in a server's metadata matched the server it connected to. A malicious server could steer the OAuth flow to a real authorization server and walk away with a token for the real resource.
• LiteLLM: authentication bypass. The gateway's MCP endpoint let a request with a made-up Authorization header fall through to an OAuth2 passthrough path, which treated it as authenticated. CISA added it to the Known Exploited Vulnerabilities catalog on September 2, 2026.
• Agent-to-agent prompt injection. Ars Technica reported research in which one compromised internal agent passed injected instructions to other agents that trusted it, leading in some cases to server-side request forgery (SSRF), for example through an HTTP client in Google's MCP Toolbox for Databases that followed redirects without checking where they went.

The 2025 bugs are still worth knowing. mcp-remote, a popular OAuth proxy, passed a server-supplied authorization URL to the system shell (CVE-2025-6514), and early MCP Inspector versions ran an unauthenticated proxy that could run commands (CVE-2025-49596). The pattern hasn't changed: anything an MCP server sends you is untrusted input, including metadata, URLs and tool output.

I reproduced the MCP SDK OAuth flaw locally

I built a three-process lab on 127.0.0.1 with Node.js 24.22:

• a legitimate authorization server on port 4401, the only one the client should ever send its secret to;
• a malicious MCP server on port 4402, which answers every request with 401 and protected resource metadata naming the attacker's authorization server;
• the attacker's authorization server on port 4403, which logs every request it receives.

The client used StreamableHTTPClientTransport with an auth provider in four configurations. In each one it connected to the malicious server while holding credentials meant only for the legitimate authorization server. I ran it against the last vulnerable release (1.30.1), the first fixed release (1.31.0) and the current release (1.32.1).

Here's what the attacker's authorization server received from the vulnerable client in the stored-token case. These are the lab's dummy values:

{"authorization":"Basic ZGVtby1jbGllbnQ6ZGVtby1zZWNyZXQtZm9yLUxFR0lULUFTLW9ubHk=",
 "body":{"grant_type":"refresh_token","refresh_token":"legit-refresh-token-123",
         "resource":"http://127.0.0.1:4402/mcp"}}

The Basic header decodes to demo-client:demo-secret-for-LEGIT-AS-only. That's the client secret and a refresh token for the legitimate server, handed to the attacker.

With the fix configured, the 1.31.0 client refused before sending anything:

Error: OAuth client information is bound to authorization server http://127.0.0.1:4401
and is not presented to http://127.0.0.1:4403.

In the issuer-bound stored-token case, the attacker's server only received a metadata GET and a dynamic client registration POST containing the public client name and redirect URI. It got no secret and no token.

Two findings matter in practice:

1. Upgrading alone doesn't fix bundled providers. ClientCredentialsProvider, PrivateKeyJwtProvider, StaticPrivateKeyJwtProvider and CrossAppAccessProvider still follow the server's choice unless you pass expectedIssuer. 1.31.0 only prints a deprecation warning. The advisory says this explicitly, and my lab confirmed it on 1.32.1 too.
2. Old saved tokens stay dangerous. Credentials saved before 1.31.0 carry no issuer, so the fixed SDK still uses them as-is. Clear them so users sign in again, or add issuer to them, and rotate any secret that may have reached an untrusted server.

How do I fix the MCP SDK OAuth vulnerability?

For TypeScript clients:

npm i @modelcontextprotocol/sdk@^1.31.0    # 1.x
npm i @modelcontextprotocol/client@^2.2.0  # 2.x

import { ClientCredentialsProvider } from '@modelcontextprotocol/sdk/client/auth-extensions.js';
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamableHttp.js';

const authProvider = new ClientCredentialsProvider({
  clientId: process.env.MCP_CLIENT_ID!,
  clientSecret: process.env.MCP_CLIENT_SECRET!,
  expectedIssuer: 'https://auth.example.com', // the only AS allowed to see these credentials
});

const transport = new StreamableHTTPClientTransport(new URL('https://mcp.example.com/mcp'), { authProvider });

If you wrote your own OAuthClientProvider, store exactly what saveTokens() and saveClientInformation() receive, including the new issuer field. Return issuer from clientInformation() for pre-registered clients too. For Python, upgrade to mcp 1.30.0 or 2.2.0, and pass an explicit issuer= for machine-to-machine providers, as the WorkOS write-up of the three bugs explains.

Then:

• clear credentials saved before the upgrade, or add issuer to them;
• rotate client secrets and signing keys, and revoke tokens, for any client that may have connected to an untrusted server;
• remember that a new interactive sign-in still goes to whichever authorization server the MCP server names. Only complete sign-ins for servers you trust.

What does the MCP spec require for authorization?

The 2026-07-28 MCP authorization spec already defends against these bugs. Implementations just have to follow it:

• Resource indicators (RFC 8707). Clients MUST send resource in both the authorization and token requests, set to the MCP server's canonical URI.
• Audience validation. Servers MUST verify that a token was issued for them and reject everything else with 401. Clients MUST NOT send a server tokens issued for something else, which rules out token passthrough.
• Issuer recording (RFC 9207). Before redirecting, the client MUST record the issuer from the validated authorization server metadata and check the iss returned on the redirect. Authorization servers SHOULD send iss and advertise authorization_response_iss_parameter_supported.
• PKCE with S256, and protected resource metadata (RFC 9728) for discovery.

The companion security best practices page covers the confused deputy problem, token passthrough, SSRF (including SSRF against authorization servers), session hijacking, local server compromise, authorization URL validation, mix-up attacks, localhost redirect impersonation, client ID metadata document trust policies and scope minimization. The October SDK flaw is a mix-up attack. If you write MCP auth code, read that page end to end.

How does fabwaseem.com secure its own MCP server?

This site runs a production MCP server for its CMS. It's how these posts get published. Here is what it does, verified against the live endpoint and the code. I've left out anything that isn't already public.

Discovery and challenge

An unauthenticated tools/list returns 401 with a standard challenge pointing at protected resource metadata:

HTTP/2 401
www-authenticate: Bearer resource_metadata="https://www.fabwaseem.com/.well-known/oauth-protected-resource",
  error="invalid_token", scope="projects:read ... site:read offline_access",
  error_description="Invalid or missing bearer token"

The resource metadata names exactly one authorization server, the site itself, and the authorization server metadata at /.well-known/oauth-authorization-server advertises:

• issuer: https://www.fabwaseem.com and authorization_response_iss_parameter_supported: true
• code_challenge_methods_supported: ["S256"] only, so no plain PKCE
• grant_types_supported: ["authorization_code", "refresh_token"] with public clients (token_endpoint_auth_methods_supported: ["none"]), so there's no client secret that could leak
• dynamic client registration and client_id_metadata_document_supported: true

Tokens bound to the resource

The authorization code, the code exchange and every refresh check that the requested resource canonically matches https://www.fabwaseem.com/api/mcp. A mismatch returns invalid_target. PKCE verifiers are compared in constant time. Codes live for 5 minutes and access tokens for 1 hour. Refresh tokens rotate on every use, and reusing an old refresh token revokes the whole token family. Tokens, codes and refresh tokens are stored only as peppered SHA-256 hashes.

SSRF-safe client metadata fetching

Client ID metadata documents mean the server fetches a URL the client chooses. That's an SSRF risk, and the spec calls it out. The fetcher only accepts https: URLs with no credentials or fragments. It resolves DNS itself and rejects private and loopback addresses, then connects to the checked IP while keeping the original hostname for TLS, so DNS rebinding can't swap in an internal address after the check. It also caps the response at 64 KB and times out after 5 seconds. Redirect URIs must match exactly, and plain http: is only allowed for loopback hosts.

Per-tool scopes, default deny

Every one of the 23 tools maps to one scope (posts:read, posts:write, posts:publish, history:undo and so on). A create_post or update_post call that would publish the post additionally requires posts:publish. The gate is three rules:

// A tool without a scope mapping needs full access, so a new tool is never
// callable by a scoped token by accident.
const required = scopesForToolCall(name, args) ?? [ADMIN_SCOPE]
if (scopeAllows(granted, required)) return null
return { id, required, message: `Insufficient scope: ${required.join(" ")} is required` }

• Unmapped tools require admin. Adding a tool can't silently widen what an existing token can do.
• Batches are all-or-nothing. Every message in a JSON-RPC batch is checked, and one denied call blocks the whole batch.
• Some scopes are never implied. history:undo must be granted explicitly, even to an admin token. When I listed tools with this site's full-access token, the server returned 22 of 23, hiding undo_history.

tools/list is filtered too, so a scoped client only sees tools it can call. That's a security and a context-window win. Measured on the live server and counted with the o200k_base tokenizer:

A read-only agent loads 91% fewer tool-definition tokens and can't even see the write tools. For more on how much context MCP tool definitions cost, see MCP vs API.

Audit log and undo

Every write, whether from MCP, REST or the admin UI, records an audit event with:

• the actor type (OAuth client, API token or admin session) and the token or client name;
• the source, the tool or route, and the action;
• the entity and a field-level diff;
• a request ID, a hashed IP and the user agent.

Raw IPs and token values are never stored, and the caller's scopes are used for the authorization check but never persisted. Events can be listed with history:read and reversed with history:undo. The 15 unit tests that cover the scope gate, PKCE, redirect matching, resource binding and private-address blocking all pass.

MCP security checklist

If you build MCP clients:

1. Upgrade to @modelcontextprotocol/sdk 1.31.0+, @modelcontextprotocol/client 2.2.0+, mcp (Python) 1.30.0+ or 2.2.0+, or rmcp 2.0.0+.
2. Pass expectedIssuer (TypeScript) or issuer= (Python) to every machine-to-machine provider.
3. Clear or migrate credentials saved before the upgrade. Rotate secrets that may have reached untrusted servers.
4. Treat server metadata, authorization URLs and tool output as untrusted input. Never pass a server-supplied URL to a shell, which is the mcp-remote lesson.
5. Ask the user before connecting to a new server, and before anything destructive.

If you run MCP servers:

1. Validate the token audience against your canonical resource URI. Never forward client tokens to upstream APIs.
2. Support PKCE S256 only. Send iss on redirects and advertise it.
3. Give every tool a scope, deny unmapped tools, filter tools/list, and check every message in a batch.
4. Prefer public clients with PKCE over client secrets you'd have to protect.
5. Rotate refresh tokens, detect reuse, and store only hashes of tokens and codes.
6. Fetch client metadata URLs with SSRF protections: an HTTPS allow-list, IP checks after DNS, pinned connections, size and time limits.
7. Log every write with the actor, tool and diff, and make writes reversible.
8. Rate-limit token and registration endpoints, and keep the MCP Inspector and other dev proxies off public networks.

Common MCP security mistakes

• Upgrading the SDK and stopping there. Bundled credential providers stayed vulnerable in my lab on 1.31.0 and 1.32.1 until I passed expectedIssuer.
• Keeping pre-fix tokens. Tokens and client information saved without issuer are still sent wherever the server points.
• Token passthrough. Accepting any bearer token, or forwarding it to another API, breaks the audience boundary the spec relies on.
• Scopes on paper only. A read scope means nothing if tools/call doesn't check it, or if a new tool is open to every token by default.
• Trusting tool output. Text returned by a tool can carry instructions. Agents that chain tools or other agents need their own guardrails, as the Ars Technica research showed.
• Exposing dev tooling. An MCP Inspector or local proxy listening on a public interface is a remote shell waiting to happen.

Related: building your own server? Start with how to build an MCP server in Next.js and stateless MCP servers under the 2026-07-28 spec.

Key takeaways

• Both official MCP SDKs had a high-severity OAuth flaw (CVE-2026-104850 in TypeScript, GHSA-qx49-fqc8-xw99 in Python). A malicious MCP server could collect a client's secret or refresh token.
• Fixed versions are TypeScript SDK 1.31.0 and client 2.2.0, Python mcp 1.30.0 and 2.2.0, and rmcp 2.0.0 for the related Rust bug.
• In my local lab, 1.30.1 leaked credentials in all four configurations. 1.31.0 and 1.32.1 blocked the leak only with expectedIssuer or issuer-bound stored credentials.
• Upgrading is not enough. Pass expectedIssuer, clear tokens saved before the fix, and rotate secrets that may have leaked.
• The MCP spec already requires resource indicators, audience validation, PKCE and issuer checks. Most incidents come from skipping them.
• On the server side, scope every tool, deny unmapped tools by default, filter tools/list and check batches. A read-only token on this site sees 4 tools and 680 tokens instead of 22 and 7,778.
• Log every write with the actor, tool and diff, and make it undoable. Store only hashes of tokens and IPs.

nestjs-prisma-starter is the base I use for that layer. It's NestJS 11 with Prisma 7 on PostgreSQL. This post walks through it one piece at a time, using the real code from the repository, and it's honest about the parts that still need finishing before production.

Running it locally

You need Node.js and a PostgreSQL database. The repository ships a .env.example with the two variables that are required, DATABASE_URL and COOKIE_SECRET:

git clone https://github.com/fabwaseem/nestjs-prisma-starter.git
cd nestjs-prisma-starter
npm install
cp .env.example .env
npx prisma generate
npm run start:dev

Outside production the app serves Swagger at /api/docs, and GET /health tells you straight away whether the database connection works. Generate a long random value for COOKIE_SECRET rather than keeping the placeholder, because it signs your cookies.

The bootstrap order matters

Everything security-related is wired in src/main.ts, in a deliberate order: body limits, then Helmet, CORS, cookies, CSRF and compression, then the global prefix, versioning, validation, the exception filter and logging. Middleware in Express runs in registration order, so a request is size-checked and given security headers before any route code sees it.

The app is created with buffered logs so that early startup messages go through Pino instead of Nest's default console logger:

const app = await NestFactory.create(AppModule, {
  bufferLogs: true,
});

const configService = app.get(ConfigService);
app.useLogger(app.get(Logger));

app.use(express.json({ limit: '10mb' }));
app.use(express.urlencoded({ extended: true, limit: '10mb' }));

The 10 MB body limit is generous for a JSON API. If your endpoints only take small payloads, lowering it is the cheapest denial-of-service protection you'll ever add.

Helmet: security headers

Helmet sets the response headers browsers use to lock a site down. The starter configures it explicitly instead of relying on defaults:

app.use(
  helmet({
    contentSecurityPolicy: {
      directives: {
        defaultSrc: [`'self'`],
        styleSrc: [`'self'`, `'unsafe-inline'`],
        scriptSrc: [`'self'`, `'unsafe-inline'`],
        imgSrc: [`'self'`, 'data:', 'https:'],
      },
    },
    crossOriginEmbedderPolicy: process.env.NODE_ENV === 'production',
    crossOriginResourcePolicy: { policy: 'cross-origin' },
    hsts: {
      maxAge: 31536000,
      includeSubDomains: true,
      preload: true,
    },
    noSniff: true,
    referrerPolicy: { policy: 'strict-origin-when-cross-origin' },
    xssFilter: true,
    frameguard: { action: 'deny' },
  }),
);

A few of these deserve a comment:

• HSTS is set for one year (31536000 seconds) with includeSubDomains and preload. Preload is a commitment: once your domain is on the browser preload list, every subdomain must serve HTTPS. Turn it on only when that's true.
• CSP allows 'unsafe-inline' for scripts and styles. A pure JSON API rarely serves HTML, so the policy mostly matters for the Swagger UI page in development. If you serve HTML from this app in production, tighten scriptSrc.
• crossOriginResourcePolicy: 'cross-origin' lets other origins load responses, which an API consumed by a separate frontend usually needs.
• frameguard: deny stops the responses from being framed, which closes off clickjacking.

CORS: an allow-list with credentials

CORS is driven by an ALLOWED_ORIGINS environment variable, a comma-separated list:

const allowedOrigins = configService
  .get<string>('ALLOWED_ORIGINS')
  ?.split(',') || ['http://localhost:3000', 'http://localhost:4200'];

app.enableCors({
  origin: (origin, callback) => {
    if (
      !origin ||
      allowedOrigins.indexOf(origin) !== -1 ||
      allowedOrigins.includes('*')
    ) {
      callback(null, true);
    } else {
      callback(new Error('Not allowed by CORS'));
    }
  },
  credentials: true,
  methods: ['GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'OPTIONS'],
  allowedHeaders: [
    'Content-Type',
    'Authorization',
    'X-Requested-With',
    'Accept',
    'X-CSRF-Token',
  ],
  exposedHeaders: ['X-Total-Count', 'X-Page-Count', 'Set-Cookie'],
  maxAge: 3600,
});

Three behaviors to know. A request with no Origin header is allowed, which is what curl, server-to-server calls and health checks send; CORS is a browser control, not authentication. Because credentials: true is on, the origin function echoes the specific origin back instead of *, so browsers will send cookies. And a * entry in the list allows every origin while still sending credentials, so never put * in production config. Also note that split(',') doesn't trim, so a.com, b.com with a space won't match b.com.

X-CSRF-Token is in the allowed headers, which is the hook for the next piece.

Cookies and CSRF

CSRF only matters when the browser sends credentials automatically, which means cookie-based sessions. Bearer tokens in an Authorization header aren't sent automatically, so the starter makes CSRF opt-in with CSRF_ENABLED=true:

app.use(cookieParser(configService.get<string>('COOKIE_SECRET')));

if (configService.get<string>('CSRF_ENABLED') === 'true') {
  app.use(
    csurf({
      cookie: {
        httpOnly: true,
        secure: process.env.NODE_ENV === 'production',
        sameSite: 'strict',
      },
    }),
  );

  app.use((req: any, res: any, next: any) => {
    res.cookie('XSRF-TOKEN', req.csrfToken(), {
      httpOnly: false,
      secure: process.env.NODE_ENV === 'production',
      sameSite: 'strict',
    });
    next();
  });
}

This is the classic pattern. The secret lives in an httpOnly cookie the page can't read. A second cookie, XSRF-TOKEN, is readable by JavaScript. The frontend copies its value into the X-CSRF-Token header on state-changing requests, and the middleware compares the two. Axios and Angular's HTTP client look for an XSRF-TOKEN cookie by name. Both cookies are sameSite: 'strict' and secure in production.

One caveat: the csurf package has been deprecated by its maintainers. It still works, but for a new project I'd plan a move to a maintained double-submit implementation, and keep the cookie and header names the same so the frontend doesn't change.

Compression

Responses over 1 KB are compressed, and a client can opt out by sending an x-no-compression header:

app.use(
  compression({
    filter: (req, res) => {
      if (req.headers['x-no-compression']) {
        return false;
      }
      return compression.filter(req, res);
    },
    threshold: 1024,
  }),
);

If you run behind a reverse proxy or CDN that already compresses, you can drop this and let the edge do the work.

Prefix, versioning and validation

All routes live under /api, except the health check, metrics and root. URI versioning defaults to 1, so a controller route users is served at /api/v1/users:

app.setGlobalPrefix('api', {
  exclude: ['/health', '/metrics', '/'],
});

app.enableVersioning({
  type: VersioningType.URI,
  defaultVersion: '1',
});

app.useGlobalPipes(
  new ValidationPipe({
    whitelist: true,
    forbidNonWhitelisted: true,
    transform: true,
    transformOptions: {
      enableImplicitConversion: true,
    },
    disableErrorMessages: process.env.NODE_ENV === 'production',
    validationError: {
      target: false,
      value: false,
    },
  }),
);

The validation pipe is the most important security line in the file. whitelist strips properties that have no decorator on the DTO, and forbidNonWhitelisted turns those extra properties into a 400 instead of silently dropping them. That shuts down mass assignment: a client can't add "role": "admin" to a profile update. target: false and value: false keep the submitted object and value out of error responses, and production hides validation messages entirely.

Rate limiting with three windows

Throttling is set in AppModule with @nestjs/throttler, registered as a global guard so every route is covered without decorators:

ThrottlerModule.forRoot([
  {
    name: 'short',
    ttl: 1000, // 1 second
    limit: 10, // 10 requests per second
  },
  {
    name: 'medium',
    ttl: 60000, // 1 minute
    limit: 100, // 100 requests per minute
  },
  {
    name: 'long',
    ttl: 3600000, // 1 hour
    limit: 1000, // 1000 requests per hour
  },
]),

providers: [
  {
    provide: APP_GUARD,
    useClass: ThrottlerGuard,
  },
],

The limits are deliberately loose for a general API: ten requests a second is enough for a busy single-page app, while a login or password-reset route deserves something much stricter. The throttler supports that per route with the @Throttle() decorator, and @SkipThrottle() exempts routes such as internal webhooks.

Three named windows let you allow short bursts while capping sustained traffic. Every request has to pass all three. The default storage is in memory, which is per process, so with several instances behind a load balancer each one counts separately. That's the point where you add a shared store such as Redis. Behind a proxy, also make sure Express trusts it, or every client will appear to share the proxy's IP.

Structured logging with Pino

nestjs-pino replaces Nest's logger with Pino, configured in AppModule. In development it pretty-prints on one line. In production it writes JSON at info level:

serializers: {
  req: (req) => ({
    id: req.id,
    method: req.method,
    url: req.url,
    remoteAddress: req.remoteAddress,
    remotePort: req.remotePort,
  }),
  res: (res) => ({
    statusCode: res.statusCode,
  }),
},
autoLogging: {
  ignore: (req) => req.url === '/health' || req.url === '/metrics',
},
customLogLevel: (req, res, err) => {
  if (res.statusCode >= 400 && res.statusCode < 500) {
    return 'warn';
  } else if (res.statusCode >= 500 || err) {
    return 'error';
  }
  return 'info';
},

The serializers are the security feature here. By listing the fields to keep, they leave out headers entirely, so Authorization tokens and session cookies never reach your log storage. Health and metrics checks are skipped so uptime probes don't drown real traffic, and the level follows the status code, so a 4xx is a warning and a 5xx is an error you can alert on.

The starter also registers a LoggingInterceptor that logs method, URL, status, size, response time, IP and user agent. With pino-http already logging every request, that's two lines per request. In production I'd keep one of them.

The Prisma error filter

The global exception filter turns every error into one response shape: statusCode, timestamp, path and message. It also translates Prisma's error codes into HTTP statuses:

if (exception instanceof Prisma.PrismaClientKnownRequestError) {
  status = HttpStatus.BAD_REQUEST;
  errorName = 'PrismaError';

  switch (exception.code) {
    case 'P2002':
      status = HttpStatus.CONFLICT;
      message = `Unique constraint failed on ${exception.meta?.target}`;
      break;
    case 'P2025':
      status = HttpStatus.NOT_FOUND;
      message = 'Record not found';
      break;
    case 'P2003':
      status = HttpStatus.BAD_REQUEST;
      message = 'Foreign key constraint failed';
      break;
    default:
      message = exception.message;
  }
} else if (exception instanceof Prisma.PrismaClientValidationError) {
  status = HttpStatus.BAD_REQUEST;
  message = 'Validation error';
  errorName = 'PrismaValidationError';
}

So a duplicate email becomes a 409, a missing record a 404 and a broken relation a 400, instead of a generic 500. The response hides details for 500s in production:

response.status(status).json({
  statusCode: status,
  timestamp: new Date().toISOString(),
  path: request.url,
  message:
    process.env.NODE_ENV === 'production' && status === 500
      ? 'An internal error occurred'
      : message,
});

Look at the default branch, though. Any other known Prisma error returns exception.message with a 400, and that masking only applies to 500s. Prisma messages can describe your tables and query, so in production I'd map unknown codes to a generic message as well. The same applies to the plain Error branch: its status stays 500, so it is masked, but a 400 built from a raw message is not.

The filter logs one line per error with the method, URL, status and error name. The stack trace argument is commented out in the code, which keeps logs short but makes a 500 harder to debug. I'd log the stack for 5xx responses only, since that's where you need it.

Environment validation, Prisma 7 and health

Configuration is validated at startup with Joi. DATABASE_URL and COOKIE_SECRET are required, and abortEarly: false reports every missing variable at once instead of one per restart:

export const validationSchema = Joi.object({
  NODE_ENV: Joi.string()
    .valid('development', 'production', 'test', 'staging')
    .default('development'),
  PORT: Joi.number().default(3000),
  HOST: Joi.string().default('0.0.0.0'),
  ALLOWED_ORIGINS: Joi.string().default('http://localhost:3000'),
  COOKIE_SECRET: Joi.string().required(),
  CSRF_ENABLED: Joi.string().valid('true', 'false').default('false'),
  DATABASE_URL: Joi.string().required(),
  THROTTLE_TTL: Joi.number().default(60),
  THROTTLE_LIMIT: Joi.number().default(10),
});

The Prisma side uses Prisma 7's driver adapter. The client is generated into src/generated/prisma, the connection URL lives in prisma.config.ts, and the service passes a PrismaPg adapter. It logs queries only outside production:

constructor() {
  const adapter = new PrismaPg({
    connectionString: process.env.DATABASE_URL as string,
  });
  super({
    adapter,
    log:
      process.env.NODE_ENV === 'production'
        ? ['error', 'warn']
        : ['query', 'info', 'warn', 'error'],
  });
}

PrismaModule is global, so any feature module can inject PrismaService. There's also a cleanDatabase() helper for tests that refuses to run in production. GET /health runs SELECT 1 and reports database: connected or error, along with uptime and environment, which is what a container orchestrator or uptime monitor needs.

What to finish before production

The core security layer above is real and working. A few things in the repository are still behind the README, and you should know them before you build on it:

• The README describes a Users module, a seed script, a docker-compose.yml and prisma:* npm scripts. Those aren't in the code yet, so use npx prisma generate and npx prisma migrate dev directly.
• THROTTLE_TTL and THROTTLE_LIMIT are validated but not used. The throttler limits are hard-coded in AppModule. Switching to ThrottlerModule.forRootAsync with ConfigService would make them configurable.
• The e2e test imports src/app.module, but the module lives at src/modules/app/app.module.ts, so the test needs its import fixed.
• enableShutdownHooks() and the manual SIGTERM/SIGINT handlers both close the app. One of them is enough.

None of these touch the request path, but they're the difference between a starter and a template you can clone and ship.

Takeaway

Security in a NestJS API is mostly configuration, and it belongs in one place you can read top to bottom. Helmet for headers, an explicit CORS allow-list, opt-in CSRF for cookie sessions, a strict validation pipe, layered rate limits, header-free structured logs and an exception filter that speaks HTTP instead of SQL. The code is on GitHub. If you're upgrading the framework underneath it, my NestJS 12 migration guide covers that step.

The part I care about most is that every tool is two things at once. It is a page a person can open, and it is a tool an AI agent can call over the Model Context Protocol. Both run the same function. There is no database, no account and no session.

This post walks through how that works, using the real code from the repository.

The rule: one tool, one pure function

Every tool lives in its own folder under src/tools/<slug>/:

src/tools/<slug>/
  meta.ts     name, tagline, category, FAQ, SEO copy
  tool.tsx    the client workspace a person uses
  lib/        the pure function, covered by vitest
  mcp.ts      the agent entry point, calling the same lib/

The lib/ folder is the tool. The page and the MCP handler are two thin shells around it. That split is what makes the dual surface cheap. When I fix a bug in the JSON formatter, I fix it once, and the page, the MCP server and the HTTP API all pick it up.

Here is the core of the JSON formatter's lib/format.ts:

export function formatJson(input: string, options: FormatOptions): FormatResult {
  const text = input.replace(/^\uFEFF/, "");
  const errors = issuesIn(text);
  if (errors.length > 0) {
    return { valid: false, output: "", errors, tree: null };
  }
  try {
    const value = sortIfNeeded(JSON.parse(text) as unknown, options.sortKeys);
    const output = options.minify ? JSON.stringify(value) : JSON.stringify(value, null, gap(options.indent));
    return { valid: true, output, errors: [], tree: treeOf(value, "root") };
  } catch (error) {
    const issue = issueFromSyntax(text, error);
    return { valid: false, output: "", errors: [issue], tree: null };
  }
}

Nothing in it knows about React, HTTP or MCP. It takes a string and options and returns a result object. That makes it easy to test with vitest and easy to call from anywhere.

The MCP side of a tool

The agent entry point is mcp.ts. It declares a zod input schema, a zod output schema, and a run function that calls the same formatJson:

export const inputSchema = z.object({
  json: z.string().describe("JSON text to format or validate."),
  indent: z.enum(indents).default("2"),
  sortKeys: z.boolean().default(false).describe("Sort object keys. Array order is kept."),
  minify: z.boolean().default(false),
});

async function run(input: unknown) {
  const parsed = inputSchema.parse(input);
  const result = formatJson(parsed.json, parsed);
  return outputSchema.parse({
    output: result.output,
    valid: result.valid,
    errors: result.errors.map((error) => ({ message: error.message, line: error.line, column: error.column })),
  });
}

export const tool: BenchTool = {
  slug: "json-formatter",
  name: "format_json",
  title: "Format and validate JSON",
  description: "Pretty-print or minify JSON. Returns the formatted text when it is valid, or the line and column of each syntax error.",
  openWorld: false,
  inputSchema,
  outputSchema,
  run,
};

Every tool exports the same shape, a BenchTool:

export type BenchTool = {
  slug: string;
  name: string;
  title: string;
  description: string;
  openWorld: boolean;
  inputSchema: z.ZodType;
  outputSchema: z.ZodType;
  run: (input: unknown) => Promise<unknown>;
};

Two details matter here. The output is parsed against its schema before it leaves run, so an agent never gets a shape the schema did not promise. And openWorld is an explicit flag: it is true only when the handler reaches the network, such as the llms.txt generator reading a sitemap.

A generated registry instead of a hand-kept list

With 61 tools, a hand-maintained list of imports would drift. Benchline generates it instead. pnpm gen:registry runs as the prebuild step. It reads every meta.ts and mcp.ts, validates them, and writes the registry files.

The validation is strict. It checks the metadata against zod schemas, requires unique slugs, known categories, known Lucide icons and known related tools, and requires a toolOrder that lists every tool exactly once. A tool folder without an mcp.ts fails the script, so a page can't ship without its agent entry point.

The output includes src/registry/mcp-registry.ts, which simply imports every tool:

// Generated by scripts/gen-registry.ts. Do not edit.
import { tool as llmsTxtGenerator } from "@/tools/llms-txt-generator/mcp";
import { tool as aiSeoChecker } from "@/tools/ai-seo-checker/mcp";
import { tools as ogImageGeneratorTools } from "@/tools/og-image-generator/mcp";
import { tool as jwtDecoder } from "@/tools/jwt-decoder/mcp";
// ...one line per tool

It also writes a lightweight mcp-index.ts holding just the name, title, description and openWorld flag, for pages that should list agent tools without loading every handler. Directory pages import metadata only. Each tool's UI is loaded with next/dynamic, so opening the directory doesn't download 61 tool workspaces.

The stateless MCP server

The MCP endpoint is a single Next.js route handler at src/app/mcp/route.ts:

import { handleMcpRequest } from "@/lib/mcp/handler";

export function POST(request: Request) {
  return handleMcpRequest(request);
}

export function GET() {
  return new Response("Method not allowed", {
    status: 405,
    headers: { allow: "POST", "cache-control": "no-store" },
  });
}

DELETE returns the same 405. Only POST is allowed, because there is no session to stream into or to close.

The handler builds a fresh McpServer and registers every tool from the generated registry:

function createServer() {
  const server = new McpServer({ name: "benchline", version: "1.0.0" });
  for (const tool of mcpTools) {
    server.registerTool(
      tool.name,
      {
        title: tool.title,
        description: tool.description,
        inputSchema: tool.inputSchema,
        outputSchema: tool.outputSchema,
        annotations: {
          readOnlyHint: true,
          idempotentHint: true,
          openWorldHint: tool.openWorld,
        },
      },
      async (args) => {
        try {
          const structured = tool.outputSchema.parse(await tool.run(args));
          return {
            content: [{ type: "text" as const, text: JSON.stringify(structured) }],
            structuredContent: structured as Record<string, unknown>,
          };
        } catch (error) {
          return { isError: true, content: [{ type: "text" as const, text: messageOf(error) }] };
        }
      },
    );
  }
  return server;
}

Every tool is annotated read-only and idempotent, which is true: none of them write anything. A successful call returns structuredContent plus a text copy of the same JSON, for clients that only read text. A thrown error, including a zod validation error, becomes isError: true with a readable message instead of a crashed request.

Then each request gets its own server and transport:

const server = createServer();
const transport = new WebStandardStreamableHTTPServerTransport({
  sessionIdGenerator: undefined,
  enableJsonResponse: true,
});
await server.connect(transport);
return transport.handleRequest(request);

This is the "stateless" part. sessionIdGenerator: undefined turns off sessions in the MCP TypeScript SDK, so there is no session id to store or route. enableJsonResponse: true answers with a plain JSON body instead of opening an SSE stream. Every request stands alone, so the server scales like any other route handler, and the README's curl examples work without an initialize call first:

curl -sS -X POST https://benchline.fabwaseem.com/mcp \
  -H 'content-type: application/json' \
  -H 'accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'

The web-standard transport (WebStandardStreamableHTTPServerTransport) takes a Request and returns a Response, which matches the App Router route handler signature exactly. There's no Node req/res adapter to write.

Guards in front of the server

Before the server is built, the handler checks two things.

First, the Origin header. A request without one, like curl, is fine. A request with one must match the request host, localhost, or the site URL:

function originAllowed(request: Request) {
  const origin = request.headers.get("origin");
  if (!origin) return true;
  let url: URL;
  try {
    url = new URL(origin);
  } catch {
    return false;
  }
  const host = request.headers.get("host");
  if (host && url.host === host) return true;
  if (url.hostname === "localhost" || url.hostname === "127.0.0.1") return true;
  return origin === SITE_URL;
}

Second, a rate limit of 60 requests a minute per address. The limiter is a small sliding window kept in memory:

const hits = new Map<string, number[]>();

export function rateLimit(
  key: string,
  limit = 8,
  windowMs = 10 * 60 * 1000,
  now = Date.now(),
): { ok: true } | { ok: false; retryAfter: number } {
  const recent = (hits.get(key) ?? []).filter((time) => now - time < windowMs);
  if (recent.length >= limit) {
    const retryAfter = Math.max(1, Math.ceil((recent[0]! + windowMs - now) / 1000));
    hits.set(key, recent);
    return { ok: false, retryAfter };
  }
  recent.push(now);
  hits.set(key, recent);
  return { ok: true };
}

The MCP route calls it as rateLimit(`mcp:${clientIp(request)}`, 60, 60_000), and a 429 comes back as a JSON-RPC error with a retry-after header. Being in memory, it is per instance, which is a fair trade for a free, keyless service with no database.

Same function, third surface: a JSON API

Not every client speaks MCP, so the same tool.run is exposed at POST /api/v1/tools/<slug>, with an OpenAPI 3.1 document at /openapi.json built from the zod input schemas. The route looks the tool up by slug and runs it:

const { slug } = await context.params;
const tool = mcpToolBySlug(slug);
if (!tool) {
  return Response.json({ error: "No tool uses that slug." }, { status: 404, headers: { "cache-control": "no-store" } });
}
// ...parse the body, then:
const output = tool.outputSchema.parse(await tool.run(body));
return Response.json(output, { headers: { "cache-control": "no-store" } });

It has its own 60-a-minute bucket. Because both surfaces call one run, they can't disagree about what a tool does.

When one tool needs more than one MCP call

Most folders export a single tool. The Open Graph image generator is the exception, and the generated registry imports it as tools instead. An agent can't see the visual template picker a person uses on the page, so the generator is split into two calls. list_og_templates returns the template ids, categories, defaults and field schemas. build_open_graph then takes a template id, the fields that template uses, colors, a size and optional images, and returns a PNG plus the Next.js file. The page and the agent still share the same rendering code. The only difference is that the agent asks what it can choose from before choosing.

File inputs for an agent with no disk

A person drops a file onto the page and the browser does the work. An agent calling a remote server has no local disk on that server, so the image and PDF handlers accept a file in one of two forms, url or data, never both. data is standard or URL-safe base64, or a data: URI, capped at 10 MB once decoded. url has to be public http or https, and it goes through the same SSRF checks described below. The fetch times out after 15 seconds, and a declared or streamed body over 10 MB is refused. The file is read in memory for that one call and not written to disk, which keeps the "nothing is stored" promise true on the server as well as in the browser.

Tools that touch the network

According to the README, 52 of the 61 tools run entirely in the browser. The other nine have to fetch something public: the llms.txt generator, AI SEO checker, robots.txt checker, meta tags inspector, page to Markdown, DNS lookup, sitemap validator, HTTP headers and SSL checker.

A server that fetches URLs on request is an SSRF risk, so every fetch goes through a guard in src/lib/net/ssrf.ts. parsePublicHttpUrl allows only http and https, refuses credentials in the URL, allows only ports 80 and 443, and blocks names like localhost, *.local and *.internal. Then assertPublicHost resolves the host and checks every address it returns:

for (const record of records) {
  if (isBlockedIp(record.address)) {
    throw new PublicUrlError("That host resolves to a private address.");
  }
}

isBlockedIp refuses loopback, private ranges, link-local, CGNAT (100.64.0.0/10), multicast and the IPv6 equivalents. Redirects are followed by hand, up to four, and each hop is checked again, so a public URL can't redirect into a private one. The browser-facing routes for these nine tools also get a tighter limit of 8 requests per address every 10 minutes.

Making it readable for agents and AI search

The registry feeds more than the MCP server. /llms.txt and /llms-full.txt are generated from it, so the site's index for AI crawlers lists every tool page and every agent tool, with its HTTP endpoint, and notes which calls fetch a public URL. Each tool page also carries its own FAQ and how-to steps from meta.ts. Adding a tool updates the directory, the MCP server, the API, the OpenAPI document and llms.txt in one build.

Testing every tool through the endpoint

The test suite doesn't stop at the pure functions. src/lib/mcp/endpoint.test.ts lists every registered tool through the MCP endpoint, calls every tool through it, fetches a public PDF URL while refusing a private one, and rejects an unsupported protocol version and a foreign origin. There's also scripts/live-mcp-smoke.mts, which connects to the live server with the MCP client, calls tools/list, and then calls every listed tool with a sample input, waiting 1.2 seconds between calls to stay under the rate limit.

Connecting it to your editor

Since there's no API key, setup is one URL. In Cursor's ~/.cursor/mcp.json:

{
  "mcpServers": {
    "benchline": {
      "url": "https://benchline.fabwaseem.com/mcp"
    }
  }
}

Claude Desktop uses the same block. VS Code takes "type": "http" with the same URL, and a client that can only launch local commands can wrap it with npx -y mcp-remote https://benchline.fabwaseem.com/mcp. The full setup page is at /mcp/guide.

What I'd keep if I started again

• Put the logic in a pure function first. The page, the MCP tool and the HTTP endpoint all became small once lib/ existed.

• Generate the registry. A build-time check that fails when a tool has no mcp.ts keeps the two surfaces in step.

• Go stateless unless you need sessions. For read-only, idempotent tools, a new server per request with JSON responses is simpler to host and debug than session-bound streams.

• Treat every fetch as hostile. Resolve the host, check every address, and re-check every redirect.

The stack is Next.js 16 with the App Router, React 19, strict TypeScript, Tailwind CSS v4, shadcn/ui on Base UI, zod 4, vitest and the MCP TypeScript SDK. It runs as a standalone Node 22 build in Docker. If you want to add a tool, the steps are in CONTRIBUTING.md, and you can try the live directory at benchline.fabwaseem.com.

This post walks through the pattern I use to move LLM calls out of request handlers and into a queue, using BullMQ on Redis with Node.js and TypeScript. The example is a support-ticket triage pipeline: each ticket goes to a model, comes back as a category, a priority and a summary, and gets saved. The same shape works for lead enrichment, document extraction, content moderation or any other "call a model for every row" job.

Everything below was type-checked against BullMQ 6 and run against a local Redis with a fake model that returns a 429, then a 503, then invalid JSON, then a valid answer.

Why an AI pipeline needs a queue

An LLM call is slower and less predictable than almost anything else your backend does. Four problems show up quickly:

1. Latency. A call can take seconds. Holding an HTTP request open for that long ties up the caller and times out behind proxies.
2. Rate limits. Providers cap requests and tokens per minute. Ten parallel webhooks can trip a limit that one would not.
3. Transient failures. 429s, 5xx responses and dropped connections happen, and most of them succeed on a second try.
4. Bad output. Even with JSON mode, a response can fail your schema. That is a failed attempt, not a reason to crash.

A queue answers all four. The handler adds a job and returns 202 Accepted. Workers pull jobs at a controlled rate, retry with backoff, and park anything hopeless in a dead-letter queue where a human can look at it.

The architecture

webhook ──► triageQueue (Redis) ──► Worker (concurrency 8, 50 req/min)
                                       │
                                       ├─ ok ───────────► saveTriage() upsert
                                       ├─ retryable ────► delayed, custom backoff
                                       └─ hopeless ─────► ticket-triage-dlq

Five small files: a Redis connection, the schemas, the model client, the queue, and the worker.

1. The Redis connection

// connection.ts
import { Redis } from 'ioredis';

// Workers use blocking Redis commands, so ioredis must not give up on them.
export const connection = new Redis(process.env.REDIS_URL ?? 'redis://127.0.0.1:6379', {
  maxRetriesPerRequest: null,
});

maxRetriesPerRequest: null matters. BullMQ workers block on Redis while waiting for jobs, and the ioredis default would abort those commands after a few retries during a Redis blip.

2. Schemas for the input and the model output

// schema.ts
import { z } from 'zod';

export const TicketJob = z.object({
  ticketId: z.string(),
  subject: z.string(),
  body: z.string().max(20_000),
});
export type TicketJob = z.infer<typeof TicketJob>;

// What we accept back from the model. Anything else is a failed attempt.
export const Triage = z.object({
  category: z.enum(['billing', 'bug', 'feature_request', 'account', 'other']),
  priority: z.enum(['low', 'normal', 'high', 'urgent']),
  summary: z.string().min(10).max(400),
});
export type Triage = z.infer<typeof Triage>;

Validating the job payload protects you from old jobs enqueued by an older version of the code. Validating the model output is the real point: an enum for category means downstream code never sees "Billing issue" or "payments?".

3. A model client that reports useful errors

// errors.ts
export class ModelHttpError extends Error {
  constructor(public status: number, message: string, public retryAfterMs?: number) {
    super(message);
    this.name = 'ModelHttpError';
  }
}

// Retry only what can succeed on a second try.
export function isRetryable(err: unknown): boolean {
  if (err instanceof ModelHttpError) {
    return err.status === 408 || err.status === 429 || err.status >= 500;
  }
  return err instanceof Error && /ECONN|ETIMEDOUT|fetch failed|aborted/i.test(err.message);
}

// model.ts
import { ModelHttpError } from './errors.js';

// Any OpenAI-compatible chat completions endpoint works here.
export async function callModel(prompt: string, signal: AbortSignal): Promise<string> {
  const res = await fetch(`${process.env.LLM_BASE_URL}/chat/completions`, {
    method: 'POST',
    signal,
    headers: {
      'content-type': 'application/json',
      authorization: `Bearer ${process.env.LLM_API_KEY}`,
    },
    body: JSON.stringify({
      model: process.env.LLM_MODEL,
      response_format: { type: 'json_object' },
      messages: [
        { role: 'system', content: 'You triage support tickets. Reply with JSON only.' },
        { role: 'user', content: prompt },
      ],
    }),
  });

  if (!res.ok) {
    const retryAfter = Number(res.headers.get('retry-after'));
    throw new ModelHttpError(
      res.status,
      `model returned ${res.status}`,
      Number.isFinite(retryAfter) && retryAfter > 0 ? retryAfter * 1000 : undefined,
    );
  }
  const json = (await res.json()) as { choices: { message: { content: string } }[] };
  return json.choices[0].message.content;
}

The important part is that the error carries the HTTP status and the Retry-After value. The worker needs both to decide whether and when to try again. (Retry-After can also be an HTTP date; this version only handles the seconds form, which is what model APIs usually send.)

4. The queue, with idempotent enqueueing

// queue.ts
import { Queue } from 'bullmq';
import { connection } from './connection.js';
import type { TicketJob } from './schema.js';

export const triageQueue = new Queue<TicketJob>('ticket-triage', {
  connection,
  defaultJobOptions: {
    attempts: 6,
    backoff: { type: 'llm' }, // custom strategy, defined on the worker
    removeOnComplete: { age: 24 * 3600, count: 1000 },
    removeOnFail: { age: 7 * 24 * 3600 },
  },
});

export const deadLetterQueue = new Queue('ticket-triage-dlq', { connection });

export async function enqueueTicket(ticket: TicketJob) {
  // The ticket id doubles as the job id, so a webhook delivered twice
  // does not create two jobs while the first one still exists.
  return triageQueue.add('triage', ticket, { jobId: `ticket-${ticket.ticketId}` });
}

Webhook providers retry deliveries, so duplicates are normal. With a custom jobId, BullMQ ignores an add for an id that already exists. In my test, enqueueing the same ticket twice returned the same job id and the queue held one job.

One caveat: once a job is removed by removeOnComplete, the same id can be added again. That is why the save step below is an upsert as well. Deduplication in the queue cuts wasted model calls; idempotency in the database guarantees correctness.

removeOnComplete and removeOnFail keep Redis from growing forever. Keep failed jobs longer than completed ones, because they are the ones you will want to inspect.

5. The worker: retries, backoff, rate limits

// worker.ts
import { Worker, UnrecoverableError, type Job } from 'bullmq';
import { connection } from './connection.js';
import { ModelHttpError, isRetryable } from './errors.js';
import { callModel } from './model.js';
import { TicketJob, Triage } from './schema.js';
import { deadLetterQueue } from './queue.js';
import { saveTriage } from './store.js';

const BASE_MS = 2_000;
const MAX_MS = 5 * 60_000;

function llmBackoff(attemptsMade: number, _type?: string, err?: Error): number {
  if (err instanceof ModelHttpError && err.retryAfterMs) {
    return Math.min(err.retryAfterMs, MAX_MS); // the provider told us when
  }
  const ceiling = Math.min(MAX_MS, BASE_MS * 2 ** (attemptsMade - 1));
  return Math.round(Math.random() * ceiling); // full jitter
}

async function processTicket(job: Job<TicketJob>): Promise<Triage> {
  const ticket = TicketJob.parse(job.data);

  const controller = new AbortController();
  const timer = setTimeout(() => controller.abort(), 30_000);
  let raw: string;
  try {
    raw = await callModel(
      `Subject: ${ticket.subject}\n\n${ticket.body}\n\nReturn {"category","priority","summary"}.`,
      controller.signal,
    );
  } catch (err) {
    if (!isRetryable(err)) {
      throw new UnrecoverableError(err instanceof Error ? err.message : String(err));
    }
    throw err;
  } finally {
    clearTimeout(timer);
  }

  let parsed: unknown;
  try {
    parsed = JSON.parse(raw);
  } catch {
    throw new Error('model returned invalid JSON'); // retryable: sampling varies
  }
  const result = Triage.safeParse(parsed);
  if (!result.success) {
    throw new Error(`schema mismatch: ${result.error.issues[0]?.message}`);
  }

  await job.updateProgress(90);
  await saveTriage(ticket.ticketId, result.data); // idempotent upsert
  return result.data;
}

export const worker = new Worker<TicketJob, Triage>('ticket-triage', processTicket, {
  connection,
  concurrency: 8,
  limiter: { max: 50, duration: 60_000 }, // stay under the provider's RPM
  settings: { backoffStrategy: llmBackoff },
});

A few decisions worth calling out.

Classify every error. A 400 or 401 will fail the same way six times, so the worker wraps it in UnrecoverableError, and BullMQ skips the remaining attempts. A 429, a 5xx or a network reset is thrown as a normal error and retried.

Respect Retry-After, otherwise use full jitter. When the provider says "wait one second", the custom strategy waits one second. Otherwise it picks a random delay between zero and an exponentially growing ceiling, capped at five minutes. The randomness matters. Without it, every job that hit the same 429 retries at the same instant and hits it again. BullMQ calls a custom backoffStrategy when the job's backoff type is not one of the built-in fixed or exponential types, which is why the queue sets type: 'llm'. Returning -1 from the strategy stops retrying.

Treat bad output as a retryable failure. Invalid JSON or a schema mismatch is thrown as a plain error. A second sample often comes back clean. If it keeps failing, the job runs out of attempts and lands in the dead-letter queue instead of writing garbage to the database.

Rate-limit at the worker, not in your code. limiter: { max: 50, duration: 60_000 } caps processing at 50 jobs per minute across every worker on the same queue, because the limit is tracked in Redis. Set it a little below your provider's limit. concurrency: 8 controls how many calls one process has in flight.

Always set a timeout. A hung request holds a concurrency slot forever. The AbortController turns it into an aborted error after 30 seconds, which isRetryable treats as transient.

6. A dead-letter queue for jobs that never succeed

worker.on('failed', async (job, err) => {
  if (!job) return;
  const outOfAttempts = job.attemptsMade >= (job.opts.attempts ?? 1);
  if (outOfAttempts || err.name === 'UnrecoverableError') {
    await deadLetterQueue.add('dead', {
      jobId: job.id,
      data: job.data,
      reason: err.message,
      attemptsMade: job.attemptsMade,
    });
  }
});

The worker's failed event fires on every failed attempt, including ones that will be retried, so the handler checks whether this was the last one. Only final failures go to the DLQ. From there you can alert, show them in an admin page, or re-enqueue them after fixing a prompt.

What the test run showed

I ran the pipeline against a local Redis with fetch replaced by a scripted fake:

• Ticket 1 got a 429 with Retry-After: 1, then a 503, then a response that was not JSON, then a valid answer. It succeeded on the fourth attempt, about nine seconds after it was enqueued, and was saved once.
• Ticket 2 got a 400. It failed after a single call, with no retries, and landed in the dead-letter queue with the reason model returned 400.
• Enqueueing ticket 1 twice produced one job.

That is the behavior you want in production: transient errors absorbed, permanent errors surfaced quickly, and no duplicate work.

Production checklist

• Run workers as a separate process from your web server so a slow batch never affects request latency.
• Call await worker.close() on SIGTERM so in-flight jobs finish before a deploy kills the process.
• Put Retry-After handling and error classification in one place, like isRetryable, and test it.
• Log job.id, attemptsMade and the error on every failure. Track DLQ size as a metric.
• Make every side effect idempotent: database upserts keyed by a business id, and an idempotency key on any external API that supports one.
• Version your prompts, and store the prompt version with each result, so you can re-run a slice of jobs after a prompt change.

Wrapping up

The model call is the easy part of AI automation. The work is everything around it: deciding what to retry, how long to wait, how fast to go, and where failures end up. A queue such as BullMQ gives you those controls for the cost of a Redis instance and a few hundred lines of TypeScript.

If you are building agents rather than batch pipelines, my posts on MCP vs API and building an MCP server in Next.js cover the other side of the same problem.