Loading workspace insights... Statistics interval
7 days30 daysLatest CI Pipeline Executions
fed4204e bench(memory): stabilize CodSpeed memory benchmarks (#7730)
* bench(memory): stabilize CodSpeed memory benchmarks
Memory flags on PRs were ~85% noise, from two causes:
1. Stale baselines (dominant): the Benchmarks workflow only ran on main
pushes touching packages/** or benchmarks/**. Main commits outside
those paths had no CodSpeed run, so PRs were compared against an old
base run ([^unexpected-base] on every recent codspeed-hq comment; six
late-June PRs all compared to the same June-23 run). One outlier base
measurement then replays into every PR until the next eligible push.
Fix: drop the paths filter for the push trigger + weekly cron backstop.
2. Wall-clock timers racing the single measured run: aborted-requests
(solid/vue) resolved deferred loader data on 500/750ms timeouts raced
against abort, and drained cancellation with at most one setTimeout(0);
streaming-peak staggered deferred sections by milliseconds. Under
variable runner load and instrumentation slowdown the interleaving —
and therefore the allocation sequence — differed run to run. Fix:
count 0ms timers-phase hops (event-loop turns) instead of milliseconds
everywhere async ordering must be staged, and drain aborts with a
fixed 8-turn settlement barrier.
Local before/after evidence (8 fresh-process runs per config, CodSpeed V8
determinism flags, idle + CPU-stressed; kernel peak RSS spread):
aborted-requests/solid idle 11.2MB -> 1.8MB (GC counts bit-identical)
aborted-requests/solid stress 7.7MB -> 2.5MB
streaming-peak/solid idle 6.2MB -> 2.5MB
request-churn/solid (untouched control): unchanged, confirming harness
streaming-peak/vue keeps ~2% proxy-level variance at any tick count; its
PR flags were the identical -3.2% in all three occurrences, i.e. the
stale-base artifact rather than per-run jitter.
Bench names and workload shapes are unchanged; absolute values will step
once on the CodSpeed dashboard from the removed cross-iteration overlap
(e.g. aborted-requests/solid retained 15.6MB -> 2.5MB per invocation).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): exclude docs/examples/e2e from main benchmark runs, drop cron
paths-ignore instead of no filter: docs/examples/e2e-only pushes cannot
affect benchmark results, and falling back across such a commit is safe
because the previous run's benchmark-relevant code is identical. The
weekly cron backstop is unnecessary for the same reason.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* bench(memory): pin GC between iterations in peak-shape scenarios
A/A runs of the real memory instrument showed the two big-payload peak
scenarios were still unstable with no timers involved: whether V8
collects iteration i's garbage before iteration i+1 allocates its
payload is not reproducible run to run, so the measured peak flipped by
a whole payload (serialization-payload solid: 6.8-9.2 MB across
identical runs; peak-large-page: 3.5-3.7 MB).
Peak scenarios measure the footprint of a single request, not
accumulation, so pin the collection points: settle two event-loop turns
(so trailing renderer/stream teardown is included) and force a GC after
each iteration. Peak then deterministically measures
max(single-request footprint). Churn scenarios are unchanged — their
signal is accumulation, which a forced GC would mask; the README rule
now spells out both sides.
A/A validation (3 identical instrumented runs, solid suite):
serialization-payload 6.8-9.2 MB -> 3.5-3.6 MB
peak-large-page 3.5-3.7 MB -> 754-757 KB (0.45% spread)
streaming-peak 38.5-39.5 -> 30.3/30.3/30.3 MB
churn scenarios unchanged (aborted-requests 1.4 MB x3)
Absolute values step because cross-iteration garbage no longer inflates
the peak; the new values are the per-request footprint the scenarios
were designed to track.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>