web-stream-kit

A zero-dependency WHATWG stream transformation and binary utility toolkit.

Overview

web-stream-kit provides lightweight, modular, and composable stream transformers and binary utilities targeting the WinterCG / Ecma TC55 Minimum Common Web Platform API baseline.

It relies strictly on universal primitives (ReadableStream, TransformStream, WritableStream, Uint8Array, TextEncoder, and TextDecoder) without binding to Node.js built-ins (node:stream, node:buffer).

Runtime Compatibility

Browsers: Modern Evergreen (Chrome, Firefox, Safari, Edge)
Node.js: 18.0.0 and above
Edge & Serverless: Cloudflare Workers, Fastly Compute, Vercel Edge, AWS Lambda
Alternative Runtimes: Deno, Bun

Core Features

Text Transforms: Delimiter-based line parsing (DelimiterStream) and spec-compliant Server-Sent Events encoding/decoding (ServerSentEventParseStream, ServerSentEventEncodeStream).
JSON Transforms: Fast newline-delimited JSON line parsing (NDJsonParseStream) and full-stream payload parsing (JsonParseStream).
Pipeline Controls: Backpressure-preserving batching (BatchStream), token-bucket rate limiting (ThrottleStream), and sync/async item transformers (MapStream, FilterStream).
Buffer Utilities: Memory-conscious Uint8Array aggregation and sequence search (concatUint8Arrays, indexOfPattern).

Error Handling & Backpressure

All streams propagate errors directly to their downstream consumers via controller.error(). Internal queues strictly honor standard backpressure semantics using the underlying stream controller's desiredSize buffer controls.

Example: Pipeline Composition

import {
  DelimiterStream,
  NDJsonParseStream,
  BatchStream,
} from "web-stream-kit";

interface LogEntry {
  level: string;
  message: string;
}

const response = await fetch("[https://api.example.com/logs/stream](https://api.example.com/logs/stream)");

const pipeline = response.body!
  .pipeThrough(new TextDecoderStream())
  .pipeThrough(new DelimiterStream())
  .pipeThrough(new NDJsonParseStream<LogEntry>())
  .pipeThrough(new BatchStream<LogEntry>({ batchSize: 50, maxWaitMs: 1000 }));

for await (const batch of pipeline) {
  console.log(`Received batch of ${batch.length} logs`);
}

Description

Zero-dependency WHATWG stream transformers and binary utilities compliant with the WinterCG / Ecma TC55 Minimum Common Web Platform API standard.

Designed to run natively across all modern JavaScript runtimes including Node.js (18+), Bun, Deno, Cloudflare Workers, Fastly Compute, and modern web browsers.

Example

import {
  DelimiterStream,
  NDJsonParseStream,
  BatchStream,
} from "web-stream-kit";

const processed = readable
  .pipeThrough(new TextDecoderStream())
  .pipeThrough(new DelimiterStream())
  .pipeThrough(new NDJsonParseStream())
  .pipeThrough(new BatchStream({ batchSize: 50 }));