Hermes tool concurrency on NVIDIA DGX Spark
2026-10-19. Codex, GPT-6 (exact variant unavailable), session 01a1556c-9e42-79b1-ae36-d9ddb0818ac1.

Hardware: one NVIDIA DGX Spark, GB10, 128 GB unified memory, 20 ARM CPU cores. Ubuntu 24.04.5; NVIDIA driver 580.178.04; Docker 29.6.2.
Model: nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4, text input, LiLiCorr speculative decoding with seven draft tokens. vLLM 0.32.0. Serving configuration held fixed: max-model-len 131072, max-num-seqs 4, gpu-memory-utilization 0.75, max-num-batched-tokens 8192; flashinfer Mamba backend, float16 Mamba state cache with stochastic rounding; prefix-match-unit and block-size 64, shared prefix checkpoint enabled; qwen3_coder tool parser and nemotron_v3 reasoning parser.
Hermes v0.22.2, commit d923a5eb487a17c0a68801bba9ef0f56c81040a8. Medium reasoning; compaction threshold 104000 tokens. Fresh CLI sessions; terminal/file tools only; no child delegation, cloud fallback or Kanban dispatch. Each job has its own nonpersistent Docker sandbox with four-CPU quota and 4 GiB memory maximum.

Workload: repair a small Python function that summarizes settlement records by currency in integer cents. Ignore every occurrence of an ID after its first, regardless of status. Include settled records only. Charges add, refunds subtract, currencies remain separate and inputs must not mutate. Preserve the JSON CLI. Agents read the contract and files, run failing tests, modify the implementation, rerun tests, generate result.json using the CLI, and write a report. Each job has 35 randomized records including duplicates; six visible unit tests. Test/fixture/contract hashes detect unauthorized edits. The driver independently reruns tests and calculates expected JSON. After timing, independent CLI execution and 33 extra cases (including conflicting duplicates, pending-first duplicates and randomized mixed-currency records) determine final validity. No test says a model is generally correct at coding.

Planned experiment: two synchronized bursts at 1, 2, 4, 8 and 16 jobs, each with zero or eight seconds of synthetic sleep before each test invocation. A compliant job invokes the test service at least twice. The injected wait is not actual network latency. Job counts increase in order; delay order reverses in the second repeat. Reused isolated Hermes data roots have dependencies prepared before timing; fresh session, process and sandbox startup remain timed. Each agent has a 20-turn / 600-second budget and a driver timeout at 660 seconds. The server is not restarted between bursts, and prefix caching remains enabled. Similar tasks may share prefixes. These are bursts, not a steady-state scheduler or a diverse repository workload.

Metrics: validated jobs per minute equals successful jobs divided by burst wall time, ending at the last job's process exit. Tables and main plots use the median of the two burst statistics; whiskers show observed repeat range, not confidence intervals. Median and empirical p95 elapsed time include all launched jobs, even failed or cancelled jobs; they are not successful-completion latency for a failed job. Small-sample p95 is not a production tail-latency guarantee. Input/output token usage and API status come from the model API. vLLM metrics, system memory, GPU temperature, Linux pressure stalls and swap page counters are sampled independently. Existing swap occupancy is not evidence of active thrashing.

Interference: the first delayed one-job burst had an extra unattributed server request outside the benchmark proxy. Before the larger results completed it was excluded from primary aggregation and scheduled for a replacement after the main series. Original measurements remain preserved. Server access starts are audited against proxy starts with a half-second matching window. Absence of an unmatched request does not prove all background host activity was absent.

Timing limitation: the proxy timestamps content and tool-call deltas plus legacy reasoning_content, but vLLM 0.32.0 streams reasoning in a field named reasoning. Missing reasoning timestamps cannot be recovered from these metadata. No model time-to-first-token claim is made from the proxy. The overlap timeline shows only observed content/tool output; its pending interval can include reasoning, prefill and queueing. This understates full response-generation overlap and is not a GPU utilization trace. Whole-job latency, throughput, API status, usage and server telemetry are unaffected. The original recorder was kept unchanged throughout this series.

Limits: short agent histories, small Python jobs, controlled test waits and at most two repetitions per condition. The 131K context value is a ceiling, not actual history length. This is not a test of 100K histories, compaction, heavy builds, other models, or Hermes API/Kanban admission. The earlier raw-inference experiment used different context, active-sequence and reasoning settings; it is preserved separately and does not give a controlled estimate of Hermes overhead.

Sources:
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4
https://github.com/NousResearch/hermes-agent/tree/d923a5eb487a17c0a68801bba9ef0f56c81040a8
https://docs.vllm.ai/en/latest/features/automatic_prefix_caching/
https://docs.vllm.ai/en/latest/configuration/optimization/#preemption
https://github.com/vllm-project/vllm/blob/v0.32.0/vllm/entrypoints/generate/base/protocol.py#L325

Admission-cap reproduction: probe_api_admission_gap.py uses unchanged handler/counting functions from the pinned Hermes commit, with controlled HTTP, database and worker stand-ins. It is not a live HTTP test. Fresh runs at cap one yield 202/429; pausing the existing-session owner lookup lets two resumed submissions yield 202/202. To reproduce with a checkout of the pinned Hermes source, set HERMES_SOURCE_DIR to that checkout and run the script with standard Python 3. It verifies source hashes and writes api-admission-gap.json. No model or gateway is started.

Validation categories: primary throughput requires both artifact correctness and the requested failing-before-edit / passing-after-edit workflow plus a report. Artifact-valid jobs that omit a required step are recorded separately as workflow-only failures. The aggregate data include artifact_valid_jobs_per_min for sensitivity analysis. An association with a concurrency condition does not establish that concurrency caused any instruction-following failure. Independent checks after process exit are outside the timing interval.

Stopping rule: stop escalation on an API error, zero validated jobs in a burst, more than eight new preemptions, GPU temperature at least 84 C, or available RAM below 6 GiB. These are experiment guards, not hardware limits. Unrun conditions have no measurements. A burst terminated by a guard or timeout remains in the results.

Process disposition: the first sixteen-job burst ended with two normal exits and fourteen driver cancellations. Four recorded BrokenPipeError stream events during cancellation triggered the predeclared error stop; the remaining three sixteen-job bursts were not run. This does not establish a model-server crash. The original validation definition checks artifacts and required workflow, not process exit; some cancelled runs can therefore validate. The primary figures preserve that metric. clean_validated_jobs_per_min additionally requires normal process exit without driver cancellation, and validated_cancelled_jobs exposes the difference. The report provides both counts.

Cancelled-stream audit coverage

The sixteen-job burst has 142 server request starts but 132 completed proxy metadata records. The proxy appends a record in its handler's finalization; the server uses daemon handler threads, so shutdown can lose records for streams still pending. Four cancelled streams recorded a broken pipe; the other ten cancelled jobs are consistent with the ten unmatched starts. Request ownership cannot be reconstructed conclusively from these records, so the mismatch does not by itself establish external interference. Per-request timing and token totals at sixteen are incomplete/lower bounds. Whole-job process timestamps, independent artifacts, and separately sampled server counters remain available. Retain the burst and disclose the gap; do not silently exclude it or assert perfect request isolation.