Population performance benchmark

Mass Forge ships an opt-in, reproducible headless benchmark for its simulation payload. It measures 10k, 100k, and 1M actorless Mass entities using the compiled high-volume effect path. It does not create Actors, render meshes, run navigation, animate VAT assets, or claim total-game capacity.

Published measurements for all supported engine baselines and their raw CSV files are in Population performance reference results.

What is measured

For each population, the commandlet records:

• batch entity creation wall time;
• the exact reflected Mass Forge fragment payload for Vital, Combat, Support, and the high-volume command mailbox;
• the observed process physical-memory change during batch creation, after preallocating the benchmark's handle array;
• mean and worst wall time for direct compiled Apply traversal;
• mean and worst wall time for mailbox submission, built-in processor execution, and their end-to-end sum;
• nanoseconds per entity and a correctness result.

One untimed warm-up pass runs before the requested measured iterations. Every direct application and every queued completion must cover the exact population and return Success; otherwise the commandlet fails.

The observed process-memory delta includes Mass chunk/allocator overhead and may include unrelated allocator reuse. The reflected payload is the deterministic capacity-planning value. Neither number includes project fragments, render resources, navigation, representation, animation, or gameplay outside this benchmark.

Run it

Maintainers should close other heavy applications and use a stable power/performance mode. Run this repository-only command from the root of a complete Mass Forge source checkout; the script and benchmark fixture are excluded from the distributable plugin package:

pwsh -NoLogo -NoProfile -File Scripts/Run-MassForgeBenchmarks.ps1 `
    -EngineRoot 'C:\Program Files\Epic Games\UE_5.7' `
    -OutputPath 'C:\Benchmarks\MassForge_UE_5_7.csv'

The script builds a strict, isolated plugin package, creates a disposable blank host project, runs the benchmark under UnrealEditor-Cmd with NullRHI, validates every requested row, copies only the CSV to the chosen output path, and removes its temporary package. Existing reports are never overwritten.

Use -Iterations to select 1–50 measured samples. The runner accepts comma-separated diagnostic populations through -Counts; it safely translates them to the commandlet's plus-delimited argument. A publishable reference run must retain 10000,100000,1000000.

Reading and comparing results

Compare like with like: identical engine patch, plugin commit, build configuration, command-line counts, iterations, hardware, OS, and power state. Use the worst sample for frame-budget risk and the mean for sustained cost. An editor commandlet is a reproducible engineering baseline, not a packaged-game frame-time promise.

Record regressions against a pinned reference report. A release should be investigated when the same machine/configuration shows either a correctness failure or a sustained timing/memory regression outside the project's documented tolerance. Do not turn one machine's measurements into a universal supported-entity guarantee.

Presentation and worker-thread boundary

This benchmark isolates Mass Forge simulation and reports wall time for the serial query/processor path. VAT, Mass Representation LOD, visibility, materials, shadows, draw calls, GPU time, navigation, and project AI require the separate showcase benchmark. Worker-thread attribution requires an Unreal Insights capture; capture the commandlet or a packaged benchmark map and report game-thread, worker-thread, and worst-frame tracks separately before making a complete game-performance claim.