Streaming and memory: profiling and batch-worker tutorial

At a glance

NextPDF renders in a single pass and never keeps a document-level Document Object Model (DOM), so input-side memory is bounded by nesting depth, not element count. This page explains the streaming model, the constraints in Architecture Decision Record (ADR)-001, and how to run the engine safely in a long-running queue worker.

Install

composer require nextpdf/core:^3

Conceptual overview

NextPDF has two write paths with different memory profiles.

The default in-memory writer composes the whole document, then serializes it. Peak memory tracks total output size. That works well for typical documents, but can be costly for very large ones.

The streaming writer serializes each page as the page is composed, then flushes it before the next page begins. The shipped engine — StreamingPdfWriter, StreamingCursor, DevNullWriter, and the WriterState enum in src/Writer/Streaming/ — is real, final, tested, and shipped since 3.1.0. It is exposed through the experimental-tier StreamingWriterInterface and CursorInterface contracts. The engine classes are internal, so depend on the contracts and let Core supply the implementation. (An earlier .ai/contracts-map.md annotation incorrectly described streaming as “contract-only / no implementation”; that stale-annotation defect is tracked in issue #610 and corrected in the B1 contract docs — the engine has shipped since 3.1.0.)

The streaming engine is designed so resident memory does not grow with page count. Each finalized page’s buffer is handed to the writer and released. The cross-reference table and the /Kids page-tree references are written to php://temp/maxmemory:0 temporary streams that spill to disk immediately instead of accumulating in the PHP heap. The serialized result is a standard page tree whose Count entry is the number of leaf nodes (page objects) descendant from a node (ISO 32000-2 §7.7.3.3) and whose Kids entry is an array of indirect references to that node’s immediate children (ISO 32000-2 §7.7.3.2). The exact memory profile is an experimental-tier property and may shift between minor releases, so do not hard-code an assumption from one measurement.

ADR-001 governs the HTML render pipeline’s memory model. The tokenizer produces a token list in one pass. The parser consumes it from left to right and emits content-stream operators into a string buffer. No persistent element tree is built: the parser holds at most one HtmlStyleState per nesting level, bounded by MAX_NESTING_DEPTH = 100, and enforces a MAX_ELEMENT_COUNT = 50_000 hard cap. The two operations that need lookahead — table column sizing and the :has() / :last-child selector family — use bounded pre-scan index arrays over the flat token list, not a retained DOM. The Phase 0 benchmark (docs/architecture/adr-001-memory-benchmark.md, executed 2026-04-06, PHP 8.5.3, memory_limit=1G) measured a 50,000-element document at 50 MB peak for the stream path versus a 4 MB partial-work retained simulation. The report attributes roughly 50 MB of that to the architecture-invariant accumulated content stream and isolates an input-side advantage of 4–5x for the stream model on that fixture. Those figures were observed on that one rig and fixture, not guaranteed.

Profile memory before you tune

Measure before you change anything. The HTML pipeline is gated by tools/perf-benchmark.php (run via composer ai:perf-check), which reports peak_memory_delta_bytes — the per-target incremental peak used as the regression axis, not the absolute process peak. The Cycle 36 baseline (docs/architecture/PERFORMANCE-BUDGETS.md §6.3, captured 2026-05-17 on an i9-13900K, 64 GB, PHP 8.5.3, opcache off) observed a 0-byte peak delta on 12 of 16 target/mode pairs. The four non-zero deltas were attributed to first-touch font-cache and trace-buffer allocations that stay constant on later renders. Read those as observed values for that rig, not as portable constants. For an ad-hoc profile of your own document, sample memory_get_peak_usage(true) before and after the render, and reset the peak with memory_reset_peak_usage() between iterations, the same way the benchmark isolates per-target cost.

Run NextPDF in a batch worker

A queue worker is a long-lived PHP process: it boots the framework once, stays resident, and handles jobs in a loop. That is what makes it fast, and it is also why memory hygiene matters. A slow leak that is invisible in a single request can accumulate across thousands of jobs. PERFORMANCE-BUDGETS §1 names this failure mode explicitly: a worker that renders many PDFs back-to-back can exhaust memory after hours even when single renders look fine.

NextPDF supports worker environments. DocumentFactory lets a worker create a fresh document for each job while sharing a process-lifetime FontRegistry and ImageRegistry, so font and image parsing happens once instead of once per job. ADR-001 records that the HTML parser is constructed per request with no static mutable state, and that future formatting-context objects must follow the same per-request scoping. The following steps configure a worker safely.

Create the registries once at process boot and reuse them for every job, following examples/14-worker-factory.php:

<?php

declare(strict_types=1);

require_once __DIR__ . '/../vendor/autoload.php';

use NextPDF\Core\DocumentFactory;
use NextPDF\Core\PdfFactory;
use NextPDF\Graphics\ImageRegistry;
use NextPDF\Typography\FontRegistry;

// Created once at process boot — not per job.
$fontRegistry  = new FontRegistry();
$imageRegistry = new ImageRegistry(maxCacheBytes: 50 * 1024 * 1024);
$documentFactory = new DocumentFactory($fontRegistry, $imageRegistry);

$factory = PdfFactory::new()
    ->withCompress(true)
    ->withDocumentFactory($documentFactory);

// Per job: a fresh document, shared registries.
$doc = $factory->create();
$doc->addPage();
$doc->setFont('helvetica', '', 11);
$doc->cell(0, 8, 'Rendered inside a worker.', newLine: true);
$doc->save('/path/to/output.pdf');

The image registry’s maxCacheBytes bounds the shared cache, so it cannot grow without limit across jobs.

Step 2 — Bound the worker lifetime

This is general process-control practice for any PHP worker, not a NextPDF engine guarantee: restart workers periodically so a long-lived process cannot accumulate memory or keep running stale code indefinitely. Both major PHP queue systems provide built-in limits and graceful restarts.

For Laravel queues (https://laravel.com/docs/12.x/queues), the queue:work command runs the worker as a long-lived process. The documented options are --memory (default 128 MB; the worker exits when its memory exceeds the limit), --max-jobs (exit after a number of jobs), and --max-time (exit after a number of seconds). The queue:restart command signals workers to exit gracefully after the current job, so a deploy or periodic timer can recycle them without interrupting an in-flight render. Laravel Horizon (https://laravel.com/docs/12.x/horizon) supervises Redis workers with an auto balancing strategy and a graceful php artisan horizon:terminate, which finishes in-flight jobs before the process monitor restarts the supervisor.

For Symfony Messenger (https://symfony.com/doc/current/messenger.html), the messenger:consume command runs forever by default. The documented limit options are --limit (handle N messages, then exit), --memory-limit (for example 128M; exit when memory reaches the limit), and --time-limit (for example 3600; exit after the interval). Symfony’s documentation recommends running the worker under Supervisor or systemd so an exited process restarts automatically, and messenger:stop-workers sets a cache flag that tells each worker to finish its current message and exit cleanly.

Step 3 — Restart on deploy

On every deploy, signal a graceful restart so workers pick up new code: php artisan queue:restart (or php artisan horizon:terminate) for Laravel, php bin/console messenger:stop-workers for Symfony. The process manager — Supervisor, systemd, or the Horizon/Octane supervisor — then starts a fresh process against the new codebase. This is general deployment practice for long-lived PHP workers and is independent of NextPDF.

Performance

The streaming path is designed to bound peak memory by flushing each completed page and spilling cross-reference and page-tree bookkeeping to disk-backed temporary streams. As a result, the resident set is intended not to grow with page count. That behavior is observed in the shipped 3.1.0 engine and pinned by its golden-baseline reproducibility tests, but it is stated as design behavior rather than a fixed number because the profile is an experimental-tier property. The HTML pipeline’s input-side memory is bounded by MAX_NESTING_DEPTH = 100 rather than element count (ADR-001). All concrete numbers on this page are tied to a dated artifact — the 2026-04-06 ADR-001 benchmark and the 2026-05-17 PERFORMANCE-BUDGETS Cycle 36 baseline — and were observed on the rigs those documents name; treat them as observations, not portable guarantees. The performance_budget of 1500 ms / 64 MB is the canvas envelope, not a contractual cap.

Security notes

The streaming cursor’s writeContent() appends bytes to the page content stream verbatim. It does not validate operator syntax. In a worker that renders caller-influenced content, never pass untrusted input to writeContent(); use writeText(), which the shipped cursor escapes for the PDF literal-string grammar. The caller owns the output stream: the engine writes to it but never closes or reopens it, so it cannot redirect output. A worker must close the handle itself after the writer’s close() returns, or it leaks a file descriptor across jobs. Sharing registries across jobs is a performance optimization, not a trust boundary: a shared ImageRegistry caches parsed images, so size its maxCacheBytes deliberately and do not assume cache isolation between tenants in a multi-tenant worker.

Conformance

Claim	Standard	Clause	Evidence
The streaming writer emits a page tree whose `Kids` entry is an array of indirect references to the node’s immediate children.	ISO 32000-2	§7.7.3.2
The streaming writer emits a `Count` entry equal to the number of leaf page objects descendant from the page tree node.	ISO 32000-2	§7.7.3.3

Clauses are paraphrased and glossary-pinned; no normative text is reproduced.