Native parallel computing

Aner separates a scientific computation from its execution context. A researcher selects a bounded local CPU allocation or an explicitly configured group of workers, while the supported computation keeps the same parameters and result contract. The first native workload is reproducible Monte Carlo estimation of π. It establishes resource budgeting, independent task execution, result accounting and progress. Supported Dense classifiers now use the same explicit context approach through distributed neural training.

This guide targets the current development engine and editor extension 0.1.33. An older executable with the same 0.1.0-dev version text may not include aner.parallel; inspect its help and build provenance. This feature does not automatically distribute arbitrary source code, mutable class instances or Tensor graphs.

Capability boundary

Single device Tensor acceleration is already documented in Tensor devices. The Dense replica extension is documented separately. GPU Monte Carlo tasks, direct collectives and model sharding are not implied by that extension. Planned capabilities on this page remain design contracts.

The following diagram preserves the initial CPU milestone boundary. Current Dense replica flow appears in the distributed neural guide.

Open the native execution diagram at full size

Start with a simple native computation

import aner.parallel

let result = parallel.monte_carlo(samples: 10000000, seed: 42)
print(result.estimate())
print(result.samples())
print(result.summary())

The simulation draws reproducible two dimensional points in the unit square and counts those within the unit quarter circle. Its estimate is 4 × hits / samples. Each point derives from the supplied seed and its global sample identity. Changing worker count, chunk boundaries or completion order does not change the sample set or integer hit count. This is a numerical simulation stream; it is unsuitable for cryptographic secrets. A π estimate has sampling error and is not an exact constant.

Omitting the execution argument uses a local context with conservative automatic CPU concurrency. A run is bounded by the requested sample count. Small workloads can finish faster with one worker because startup and scheduling cost remain part of execution.

Download the complete local serial/parallel program. It runs without cluster configuration.

Explicit local control

import aner.parallel

let execution = parallel.local(workers: 4, chunk: 65536)
let result = parallel.monte_carlo(
    samples: 10000000, seed: 42, execution: execution)
print(result.hits())
print(result.workers())
print(result.seconds())

A context is an immutable execution policy. workers: 0, or omitting workers, requests the automatic policy. A positive value requests that concurrency and fails if it exceeds the discovered allocation or implementation limit. It is never silently increased to use all CPUs. The chunk is the number of samples assigned as a task; the final chunk retains the remaining samples. Smaller chunks improve load distribution and cancellation granularity, but add scheduling and communication overhead.

parallel.inventory() returns machine readable local allocation information. Discovery distinguishes logical CPU visibility from effective affinity, quota and scheduler limits where available, and retains notes for unknown conditions. Virtual CPUs are not assumed to be dedicated physical cores. Operating system and scheduler limits still govern actual CPU time. Configuring several VMs does not prove that their underlying physical resources are independent.

The native executor uses a fixed, bounded set of reusable worker threads and a bounded pending queue. It does not create a thread for each point. The interpreter remains responsible for checked source execution; mutable user objects and shared differentiation graphs are not concurrently evaluated as arbitrary callbacks. The Monte Carlo kernel does not invoke a nested numerical thread pool. CPU concurrency for other numerical operations requires a separate supported budget contract.

The implementation admits at most 256 local CPU workers, 32 configured worker records, 10 billion samples and one million chunks. The default chunk is 65,536 samples. These are admission limits, not recommended resource settings or a substitute for a process memory quota. Choose a larger chunk if the requested count would exceed the bounded chunk ledger.

Use explicitly configured workers

import aner.parallel

let execution = parallel.cluster(config: "private.cluster")
let result = parallel.monte_carlo(
    samples: 10000000, seed: 42, execution: execution)
print(result.summary())

The complete configured worker program requires operator configuration; it is not a turnkey runnable download. private.cluster is operator maintained private configuration, not a file supplied by this public example. It names approved local or SSH workers, their public logical identities, compatible installed Aner executable paths and per worker CPU limits. Real endpoints and authentication remain private. A configuration never enrolls every machine it can discover.

The initial worker runs a bounded native scientific protocol over protected local pipes or an authenticated SSH session. It opens no public worker listener. SSH host key verification and the user's existing authentication must succeed; an unavailable machine is not converted into an unreported local computation. Each remote OS/architecture needs a compatible Aner build. Before task or analytic gradient execution, the worker handshake checks protocol, assigned CPU slots, platform identity and a SHA256 fingerprint of the same 12 file native parallel source contract. A mismatch fails before that worker contributes scientific work. Source contract identity is separate from the target specific executable hash and compiler record; matching source is not a promise of bitwise numerical equality across architectures. SSH connectivity alone does not install an engine, prove a backend capability or authorize data transfer. The initial local process and SSH transports require POSIX systems; Windows retains local native CPU computation, while its process/SSH transport is unavailable in this slice.

The initial task protocol carries scientific task parameters and sufficient statistics. Supported Dense training now uses its separate validated numerical transport contract; arbitrary source functions and optimized GPU collectives remain later work. Native tasks can be dispatched to different architectures without transferring process pointers or assuming a shared filesystem.

For collaboration, each researcher explicitly grants a CPU/device allocation and the permitted data/artifacts. Prefer coarse independent work before tightly synchronized training across a network. A private tunnel can establish reachability; it cannot remove latency or make remote memory local. A shared server's worker must remain within its assigned compute allocation.

Terminal jobs and private configuration

The native CLI exposes the same supported simulation without writing source code:

aner parallel inventory --json
aner parallel run --samples 10000000 --seed 42 --workers 1 --output simulation-serial
aner parallel run --samples 10000000 --seed 42 --output simulation-parallel
aner parallel status --job simulation-parallel --json

Use a new output directory for every run. The CLI records run-manifest.json, atomically replaced status.json snapshots and a terminal result.json when available. The manifest retains requested work, sample stream identity and local allocation inventory. A completed result must account for the exact requested samples. --quiet suppresses terminal progress; it does not change the simulation or result accounting.

For a configured group, prepare a private text file with this implemented schema. The paths and SSH alias below are placeholders to replace with approved installed executables and an enrolled SSH configuration:

ANER_CLUSTER 1
worker "Atlas-01" local "-" "/path/to/aner" 2
worker "Atlas-02" ssh "research-peer" "/path/to/aner" 2

Every record supplies a unique public Atlas identity, local or ssh transport, endpoint, absolute executable path and positive per worker CPU slots. Use one enrolled worker record per platform in this initial schema. The worker validates its requested slots against its discovered allocation. Aggregate local process slots also respect the submitting machine's allocation. A cluster needs at least one sample chunk per configured worker so every selected worker participates; remaining work is dispatched as workers become ready.

chmod 600 private.cluster
aner parallel run --samples 10000000 --seed 42 --chunk 65536 \
    --cluster private.cluster --output simulation-cluster
aner parallel status --job simulation-cluster --watch

On POSIX, configuration is an owner only regular file; job output is an owner protected directory. SSH uses verified host keys and batch authentication. Keep credentials out of this file, source code and command arguments; configure the authentication through approved SSH credentials. Real endpoints remain private. Public sample output uses Atlas identities.

POSIX systems also support a detached native CPU job with an independent monitor and cancellation:

aner parallel run --samples 100000000 --seed 42 \
    --output simulation-background --background
aner parallel status --job simulation-background --watch
aner parallel cancel --job simulation-background

--background requires a new output directory and detaches the coordinator from the submitting terminal. Its log is coordinator.log. Closing or interrupting a status monitor does not cancel that job; cancellation is the explicit owner authorized command. The job runs only while its coordinator process remains alive. On POSIX, an owner only process lock identifies the active coordinator; worker executables do not inherit that lock. A fresh status monitor reports COORDINATOR_OFFLINE after coordinator loss and retains the last confirmed snapshot. This process lock is distinct from a persisted task lease. Atomic snapshots and detachment do not provide restart recovery, automatic failover, persisted task leases or a distributed checkpoint service. Background and process/SSH transport are unavailable in the initial Windows build.

Coordination and observation

The CLI dispatcher coordinates tasks and records accepted progress. A later durable service will add persisted leases, bounded retries and checkpoint recovery. This first slice does not resume an interrupted coordinator or claim exactly once arbitrary side effects.

CPU cancellation is cooperative at bounded kernel boundaries: stop submitting work, request active workers to stop and join them. Partial accepted counts are diagnostic information, never a successful full sample count. A task exception propagates to the submitter and pending work is stopped. A process/transport failure must identify the affected worker and fail visibly; it does not silently drop that contribution.

For source calls, Dev and Prod keep the same scientific kernel, seeds, resource checks and accounting. Prod omits optional automatic inspection. Existing development reports can observe ordinary scalar values extracted after a parallel call; detailed cluster task timelines, live GPU telemetry and notebook streaming are future observability work. A screen refresh must not become a training barrier or an automatic GPU readback. Missing utilization, memory or network measurements are N/A, not zero.

Report completed work, elapsed time and observed throughput. The historical CPU only benchmark build reported analytic_training: true and distributed_neural_training: false; those archived fixture results do not establish a neural network capability. The current development engine additionally reports supported distributed Dense training through the distributed neural guide. A remaining time estimate, when implemented, requires a fixed horizon and enough completed work. It does not predict how many epochs achieve a research target.

Experimental analytic training fixture

The current CLI also includes a small native binary logistic regression fixture for validating training reductions. It generates eight procedural features plus a bias from stable sample identities, uses fixed synthetic labels, initializes nine Float64 parameters to zero and applies full batch SGD to mean binary cross entropy.

aner parallel regression --samples 65539 --steps 8 --rate 0.25 \
    --seed 42 --workers 1 --quiet --output regression-serial
aner parallel regression --samples 65539 --steps 8 --rate 0.25 \
    --seed 42 --cluster private.cluster --quiet --output regression-cluster

The same fixed samples value is the global batch on every step, not a per worker batch. Workers receive disjoint ranges with frozen parameters and return gradient sums, loss sums and valid counts. The coordinator divides the global sums by the valid count and performs one update. Unequal final chunks remain correctly weighted. Initial and final metrics are separate complete passes.

This fixture exercises native CPU training semantics; it is not a neural network, a general regression fit API or a held out evaluation. Its fit_accuracy measures agreement on the same synthetic rows used for updates and does not establish generalization. This CPU analytic fixture remains separate from the supported Dense Dataset training API, which includes fitted preprocessing and protected mini batches. Arbitrary custom objectives and durable distributed job checkpoints remain later work. Source level contexts support the simulation and the separate Dense fit; this analytic fixture is exposed through the experimental CLI.

For e updates, n rows, f features and k row chunks, total work is O((e + 2)nf), including initial/final metric passes. Chunk gradient state is O(kf), with O(f) canonical parameters and bounded worker/transport bookkeeping. This fixture fixes f = 8. Local reductions currently use 65,536 row chunks; configured worker reduction chunks can be selected explicitly subject to admission limits. Floating reduction order can differ between a local and a differently chunked distributed plan; validate declared tolerances rather than promise cross plan bitwise equality.

Three distinct forms of cooperation

Open the parallel strategies diagram at full size

Independent simulations, model search trials and separate folds can run without a gradient barrier between tasks. Their outputs still require an appropriate aggregation rule. Cross validation folds are independent fits; they are not partitions of one global optimizer update. Shared groups and held out roles remain governed by Dataset partitions.

Synchronous replicated Dense training is a different supported computation. Every device retains a complete model and processes a disjoint part of one fixed global batch. The group combines gradients and advances the optimizer once. Each replica must fit its parameters, gradients, local activations and workspaces. The Dense extension keeps optimizer state on its canonical CPU coordinator. Two computers do not become one device with the sum of their memory.

Model or optimizer state sharding can distribute persistent state. It needs distinct ownership, communication, checkpoint and failure algorithms. It is not activated by launching two replicas or changing an execution context.

Neural training contract

Open the weighted training diagram at full size

The supported Dense training group preserves the experiment: source identity, training/validation/test roles, exact sample assignments, model initialization, optimizer, loss, requested precision, global batch and stopping rule. Normalization is fitted on training rows before held out evaluation; distributed placement must not bypass these checks. Training batches are disjoint, the final partial batch is retained, and retry results cannot be counted twice.

When worker i contributes a mean gradient gᵢ from nᵢ valid samples, the global mean is:

global_gradient = sum(nᵢ × gᵢ) / sum(nᵢ)

The native Dense runtime exchanges gradient sums and valid denominators directly. Means 2 and 4 over 32 and 224 samples combine to 3.75. An unweighted worker average of 3.0 changes the algorithm. Weighted samples or masked tokens require their appropriate valid weight denominator.

All replicas start from the same model version, hold parameters fixed during local microbatch accumulation and perform one shared update. They follow the same collective order even with unequal sample counts. A lost rank aborts the affected group; a partial gradient does not become a normal update. Durable distributed restart from a committed group checkpoint remains unimplemented.

The portable Dense reducer synchronizes local device completion, exchanges typed host buffers, applies one canonical CPU update and distributes the new state. It supports mixed CPU, Metal and CUDA replicas with explicit transfer costs and one compatible group dtype. See distributed neural training for actual signatures, precision and Dataset boundaries. A later optimized homogeneous CUDA collective can exchange gradients directly over compatible approved links. The coordinator continues to manage membership and progress without necessarily relaying all gradient bytes. NVIDIA's collective communication overview describes the native primitives; Aner's adapter requires separate implementation and hardware validation.

Precision and two cooperating Macs

The existing preserve policy retains Float64 on CPU or supported CUDA operations. Metal/MPS currently uses Float32; preserving Float64 reports CPU execution. A strict Dense cluster declares one supported compute precision for every participant, with Float64 host accumulation/canonical updates. MPS Float64 is rejected by explicit replica contexts; the separate local profile can still report preserve policy CPU fallback. Precision conversion and actual placement remain visible. See Tensor devices.

Two Apple computers can be useful for independent research tasks and may improve a sufficiently large replicated training workload on an adequate link. Useful compute must exceed scheduling, host buffer copies, network exchange and synchronization costs. A smaller model or a slow link can make the combined training run slower than the fastest computer alone. Apple's Metal framework enables native local GPU work; separate Macs still have separate memory and device queues.

A larger model whose replicated state does not fit either Mac still does not fit under ordinary data parallelism. Smaller microbatches reduce activation memory, but not the need to retain replicated model and optimizer state. Model sharding is the later route to that different capacity problem.

Time and space complexity

For the implemented simulation, let n be samples, c chunk size, k = ceil(n/c), p active CPU workers and q the bounded queue capacity. Each sample has constant native work and the sufficient statistics are integer counts.

The span bound excludes thread/process startup, OS scheduling, serialized dispatch, heterogeneous workers and transport. It is a theoretical ideal rather than measured wall clock complexity. The scientific computation stores chunk statistics, not an n row point dataset. A future map returning every transformed element would necessarily retain O(n) output or stream it through another explicit contract.

The initial CPU milestone plan used the following generic replicated storage model; the current Dense path instead keeps typed optimizer state on its canonical CPU coordinator, as detailed in the distributed neural guide. For that historical plan with P parameter scalars, dtype width d and p workers, persistent model/gradient/optimizer storage is O(Pd) per worker and O(pPd) across replicas, plus batch dependent activations and workspaces. A central streaming reduction performs O(pP) arithmetic and receives O(pPd) gradient bytes per step; it can use O(Pd) accumulator space plus bounded buffers. Retaining every worker payload instead uses O(pPd). Redistribution also transfers O(pPd) aggregate parameter bytes. The actual collective bound depends on its algorithm and topology.

Transferring G bytes at measured effective bandwidth b takes at least G/b seconds, before latency and other work. A synchronous step waits for its required contributions. More devices do not remove serial dependencies, memory bandwidth limits or barriers.

Evaluate scaling honestly

Measure a one worker reference, each machine alone, the fastest eligible machine alone and the combined plan. Fix sample count, seed and algorithm for simulations; additionally fix global batch, model, precision and optimizer for training. Report at least three repetitions, median and spread, warm up, cold staging versus cached execution, end to end duration and compute/transfer/wait costs. Preserve unfavorable results.

For comparable homogeneous resources, speedup(p) = T(1)/T(p) and efficiency(p) = speedup(p)/p. Different worker counts or allocated CPUs are not proof of useful concurrency. Heterogeneous throughput predictions remain estimates until the selected combination is measured. Report validation quality and time to a fixed quality target for training, not only steps per second.

Public reports use a stable platform alias and immutable measured configuration, such as Atlas-02 / C01. A RAM, CPU/GPU, driver or engine build change retains the Atlas ID and creates a new configuration revision. A different computer receives a different ID, including two computers with the same processor generation. Private hostnames, routes and credentials are excluded from public assets.

No multi GPU, mixed device neural training or cross network speedup is implied by the CPU simulation. Each such combination requires its own capability, numerical, failure and measured performance acceptance evidence.

Measured native CPU validation

The October 12, 2026 experiment executed the compiled native CPU implementation on one Apple computer and three Ubuntu VMs. It contains 22 cases with three measured repetitions each: 66 recorded trials, plus one excluded warm up per case. Every trial completed the requested scientific work. The report includes unsuccessful scaling configurations; it does not establish GPU or general distributed neural training performance.

A visible allocation is not a measurement of dedicated physical capacity. VM host sharing, competing workload and physical topology were not established. The remote experiments intentionally reserve one compute slot per worker, including Atlas-06; they do not compare maximum all core VM performance. The coordinator runs on Atlas-01 and contributes coordination rather than an additional simulation or gradient worker in the remote groups. Its CPU and network overhead remain in the wall time. No GPU is used in these experiments.

Scientific and timing contract

The simulation requests exactly 1,000,000,000 samples with seed 42, Float64 coordinate arithmetic and integer hit/count statistics. Every local and remote repetition produced exactly 785,380,324 hits and the estimate 3.141521296. Local and individual remote commands use 1,000,000 outer samples per chunk. Cluster cases compare that chunk with 100,000,000; the native worker subdivides its assigned range into bounded internal chunks. Coarsening changes scheduling, not sample identity or the estimator.

The analytic fixture uses the same 262,147 generated rows as one fixed global batch on each of 16 SGD updates, rate 0.25 and seed 42. Eight features plus a bias give nine initially zero Float64 parameters. Initial and final full batch metric passes bring the total to 18 passes. Final loss is approximately 0.5320923849 and fit accuracy is 0.9663852724. All parameter and loss differences were within absolute tolerance 1e 10; fit accuracy differences were within 1e 12. This is fitting set agreement on a synthetic fixture, not held out accuracy or a neural benchmark.

The primary duration is an external monotonic clock around a fresh native command, including process startup, authenticated transport when used, native work and shutdown. An individual remote reference includes one SSH invocation and the native command. A distributed case includes the coordinator command and persistent worker connections. Native internal durations are retained in the downloads but have different boundaries: a local call excludes prior execution context construction, while distributed execution includes worker setup. Those internal durations are not interchangeable end to end measurements. Data staging and compilation precede timing; no compilation or test suite ran concurrently with the timed cases.

The tables give median and observed minimum: maximum seconds over three repetitions. This spread is descriptive, not a confidence interval. Memory peaks, CPU utilization, transfer bytes, per step barrier wait and effective network bandwidth were not instrumented for this experiment and are N/A. Guest memory capacity is not process memory use. A separate protected stream small line echo probe on one participating connection measured approximately 55 to 61 ms round trips after connection establishment; it is diagnostic evidence, not a bandwidth measurement or an estimate for every link.

All selected native workers used the same 12 file source contract fingerprint:

794e80a8d69285e18f7bec4d5f06cf2a8fa85f9e4d1c9202dac661861d972bea

The Apple executable and Linux executable are different architecture specific artifacts. The Linux worker was compiled with GCC 15.2, C++20, -O3, -DNDEBUG, -fno-fast-math, -ffp-contract=off and native thread support; it has no GPU dependency. Compatible source identity was validated at the worker handshake. The standalone native worker contains the supported task and analytic fixture CLI; source programs were separately checked with the full development engine. Earlier calibration builds are excluded from this report.

Local simulation scaling

Open the local scaling chart · Download the local chart as PNG

Atlas-01 / C02 executes the identical one billion sample workload:

The measured 12 slot speedup is 7.10×, with efficiency approximately 0.59. This is evidence for this native kernel and machine configuration, not a guarantee for arbitrary source loops, heterogeneous cores or neural training.

The local analytic fixture is much shorter:

Its 65,536 row reduction plan contains only five chunks per pass. Twelve configured slots therefore cannot supply twelve full independent gradient chunks, and process startup is material relative to the short native computation.

Networked CPU execution

Open the cluster timing chart · Download the cluster chart as PNG

Simulation, fixed at one billion samples and one CPU slot per selected VM:

Increasing the pair's outer chunk from one million to 100 million samples reduces median time by approximately 6.72×. The three worker coarse plan takes 5.2713 seconds, compared with 4.5376 seconds for the fastest eligible individual VM. It is still slower. Counts verify cooperation and correctness; elapsed time shows that coordination and scheduling costs outweigh added compute for these configurations.

Analytic full batch fixture, with one CPU slot per selected VM:

Every selected worker contributed accepted samples in every phase. The coarse three worker plan has only three chunks: 131,072, 131,072 and 3 rows. Atlas-06 receives the three row chunk in each of the 18 passes, totaling 54 rows, while each other worker processes 2,359,296 rows. This deliberately retained case validates unequal count weighting; it is very uneven participation and does not demonstrate three worker training capacity scaling. The dispatcher does not yet optimize chunk size from measured device or network cost.

The measured results support coarse independent tasks as the first remote strategy. A future planner should measure startup, task and link cost, then recommend a local or distributed plan while preserving an explicit forced plan experiment. The current parallel.local and parallel.cluster policies are explicit; they do not automatically predict the fastest plan. General Tensor gradient exchange and GPU collectives require their own implementation, correctness tests and measurements.

Reproducibility artifacts and validation status

Download the complete 66 native trial records and 22 case summary. Public artifacts retain Atlas identities, immutable configuration revisions, scientific parameters, per worker accepted work, source fingerprint and timing. Operator routing and credentials remain private.

The development and release builds passed the existing native suite with the new parallel tests. Focused tests cover bounded admission, exact count parity, cancellation, exceptions, child process cleanup, source/protocol mismatch, task identity and sample accounting rejection, stable weighted gradients, uncommitted step cancellation, detached coordinator readiness and monitoring after coordinator loss. Local native executor, process, training and distributed tests also underwent race and memory safety instrumentation. Linux checks ran on Atlas-02 and Atlas-03; the three worker process cases ran on Atlas-03 because Atlas-02 exposes only two CPU slots. Atlas-06 ran only the approved one slot scientific worker. Its temporary test executable was removed after the experiment.

GPU tasks, multiple GPUs, mixed Metal/CUDA neural training, general Dataset based distributed training, durable checkpoint recovery and cross network enrollment are NOT_RUN for this slice. Their design remains staged work, rather than a benchmark conclusion.