Performance — go-ndarray vs NumPy
Mirrored from
docs/perf.mdat v0.46.0.
The table is perf-table.md, generated by refbench (go-scikit/refbench v0.3.0) from perf.json; nothing in it is edited by hand. The rules are those of the go-scikit performance campaign (go-scikit/roadmap#27):
- One core on both sides.
GOMAXPROCS=1for Go, and the reference worker runs withOMP_NUM_THREADS,OPENBLAS_NUM_THREADS,MKL_NUM_THREADS,VECLIB_MAXIMUM_THREADSat 1.GOMAXPROCS=1must also be in the environment of the test binary:ndarray/fftrests on go-fft/fft, which reads it once at init (go-fft/fft#107). - The pinned reference: NumPy 2.5.4 (
bench/refs.txt, hashes included), Python 3.12.12 throughuv. The earlier page of this repository compared multicore Go with single-threaded NumPy 2.2.4 and is kept as perf-history.md for its kernel-level analysis only. - Interleaved rounds, one statistic: a round is one sample per side, alternating which runs first; the verdict is on the median with quartiles.
- Load gate: no timing verdict at a 1-minute load of 8 or more
(
-perf.runrefuses and writes nothing). - Same data: every array argument is
lo + (hi-lo) * frac(i * 0.618...), built in Go and in NumPy;TestDataParitycompares the SHA-256 of both sides' bytes (24 distinct argument sets, complex ones included, all equal), andTestRefSmokeruns every NumPy call once. - One case per public function (a function with several shapes of use has
several rows,
Array.SumAxis/axis0and/axis1), or a named exemption with its reason in perf-exempt.md. CI fails on a public function with neither.
What a row times
Each row calls the Go function once on a vector of 100 000 elements (a 1000 x
100 matrix for the axis reductions, 128 x 128 for the matrix products, 64 x 64
for linalg, 65 536 points or 256 x 256 or 32 x 32 x 32 for fft), and the
NumPy call on the same data. The result is allocated by the call on both sides;
the *Into rows write into a preallocated array on both sides (out=). The
random rows draw 100 000 values from a generator built once outside the timed
loop (default_rng(1) / DefaultRNG(1), RandomState(1), np.random.seed(1)),
so the constructor is not in the interval. Views (Reshape, Slice,
Transpose, Ravel) are views in NumPy; where Go copies, the row says so.
fft rows include the plan on both sides: ndarray/fft builds or fetches a
plan per call, numpy.fft keeps its own plan cache warm after calibration. The
scipy.fft comparison of the transforms themselves is
go-fft/fft’s.
Value parity is not checked here: each package’s own tests compare its results
with NumPy’s fixtures (random/numpy_test.go, linalg/testdata,
fft/testdata).
Running it
cd bench
GOMAXPROCS=1 go test -run TestPerf -cpu 1 . # CI mode: allocations, table, public-function inventory
GOMAXPROCS=1 go test -run TestPerf -cpu 1 -perf.run . # measure; writes docs/perf.json and docs/perf-table.md
GOMAXPROCS=1 go test -run TestPerf -perf.record . # allocation counts only (status: deferred)
go test -run 'TestDataParity|TestRefSmoke' -perf.parity . # needs uvStatus
The table is measured at 37c1b86 (the head of the perf/fftshift-one-pass
branch, clean) on 2026-11-15, on one pinned core of cfarm420 (AMD EPYC 7773X,
Zen 3), NumPy 2.5.4. The 1-minute load over the rounds was 2.62 to 3.99, and
all 256 rows are valid. It replaces the table measured at v0.41.0
(de3f41ac, 2026-11-14, load 4.12 to 5.56), and the section below lists what
differs.
185 rows are slower than NumPy, 54 are faster and 17 tie (v0.41.0: 188, 54 and 14). Every slower row is an open defect. They are tracked by cause in #118 rather than in one issue per row:
- result allocation: the allocating
Addis 3.9× slower, whileAddIntois within 5% (1.05); - elementwise operations: the unary
Ceil(6.0×),Floor(5.8×),Square(5.5×),Abs(5.5×),Neg(5.4×) andRound(4.7×) now run on vector kernels and what remains is the result allocation;Maximum,MinimumandPower(5.3–5.8×) andLog2(4.0×) are the worst of the others; - copies where NumPy returns a view or wraps the buffer (
Flatten5.7×,Copy5.8×,Imag5.2×); linalgat 64 × 64 (1.5–3.9×);- the
fftshifts, now 2.0× (they were 9.7× and 9.9×); - the
randomdraws built onlog/exp.
Faster, for example: SqrtInto (0.50), fft.FFT (0.52), Sum (0.80), Log10
(0.57). Exp (0.98) ties.
Since the v0.41.0 table
The changes below were made after v0.41.0 and are included in the table above. The ratios of this table against the v0.41.0 table (Go time / NumPy time, 10⁵ float64):
| row | v0.41.0 | now | allocations |
|---|---|---|---|
fft.FFTShift | 9.72 | 2.03 | 29 → 9 |
fft.IFFTShift | 9.90 | 2.02 | 29 → 9 |
Ravel | 566.8 | 0.36 | 4 → 3 |
Real | 565.9 | 0.58 | 4 → 3 |
Ceil | 20.4 | 5.95 | 5 |
Floor | 19.2 | 5.75 | 5 |
Round | 13.8 | 4.67 | 5 |
Abs | 11.9 | 5.48 | 5 |
Square | 11.0 | 5.51 | 5 |
Neg | 10.3 | 5.44 | 5 |
AddScalar | 8.56 | 5.40 | 8 |
Full | 8.00 | 5.51 | 5 |
Ones | 7.34 | 5.30 | 5 |
fft.RFFT2 | 1.41 | 1.17 | 31 → 30 |
fft.RFFT2With | 1.50 | 1.15 | 31 → 30 |
fft.IHFFT | 1.33 | 1.04 | 35 → 34 |
fft.IHFFTWith | 1.22 | 1.05 | 35 → 34 |
fft.RFFTN | 0.95 | 0.75 | 32 → 31 |
fft.RFFTNWith | 0.92 | 0.73 | 32 → 31 |
fft.RFFT | 0.90 | 0.75 | 30 → 29 |
fft.RFFTWith | 0.95 | 0.79 | 30 → 29 |
fft.FFTShift and fft.IFFTShift now shift in one pass (the allocation count
falls from 29 to 9), and their ratio from about 9.8 to 2.0. This change is
well above the run-to-run spread below. They remain slower than NumPy.
Ravel and Real are now faster than NumPy, and the fft real-input
transforms each save one allocation because they call x.Real(), which is a
view. The unary rows remain slower than NumPy, by about 5× instead of 10–20×:
they still allocate the result, which AddInto shows to be the main part of
the remaining gap (Add 3.85, AddInto 1.05).
Treat small ratio changes with care: the tables were measured at different
loads, and rows whose code did not change moved by as much as 0.2 to 0.3
(Sqrt 1.01 to 1.32, fft.IRFFT 0.72 to 0.74 in the third run but 1.06 in
the second, SubScalar 4.81 to 7.11 between two runs of the same code). Only
the rows above whose change comes from a known code change, and whose size
exceeds that spread, are claimed as effects: FFTShift and IFFTShift,
Ravel and Real, and the allocation counts. The other fft real-input
ratios (0.1 to 0.3 lower than at v0.41.0) are within or near that spread, so
only their allocation counts are attributed to Real, and the elementwise
ratios do not show a difference between kernels.
The measurements that follow were made at the time by go test -bench on the
same core. On 10⁵ float64 at load 2.2–2.4, over three interleaved
rounds:
The scalar forms (
AddScalar…DivScalar,AddInto…DivInto, and any Float64 operation with a one-element operand against a contiguous array of the result’s shape) no longer copy the scalar to the result’s size. A strided operand, such as a transpose, still takes the general path. They stream it through the same SIMD kernel, so the result is bit for bit the same.AddScalarwent from 183–203 µs to 95–110 µs, and from 1.6 MB to 0.8 MB per call.Full,OnesandFullOffill by doublingcopyinstead of storing one element at a time.Fullwent from 131–135 µs to 84–104 µs.OnescallsFull.FullOf, which fills the other dtypes, was not timed.Float64
Abs,Neg,Square,Floor,CeilandRoundrun on the go-simd/floats v0.23.0 kernels (above the parallel threshold, split across cores as before) instead of a scalar loop throughMap. The results are bit for bit those ofmath.Abs,-x,x*x,math.Floor,math.Ceilandmath.RoundToEven.AbsandNegkeep NaN bits exactly; forSquare,Floor,CeilandRounda signalling NaN may come back quieted (TestUnaryKernelsBitIdentical: edge table, lengths 0–33 and one above the threshold).Roundis now half to even, asnp.round; the old half away from zero isRoundHalfAway, on the scalar path (see the README).go test -bench Unary(benchmarks/unary_bench_test.go), 10⁵ float64 including the result allocation, one pinned core of cfarm420 (GOMAXPROCS=1,taskset -c 9), a test binary of v0.42.0 and one of this change, four interleaved rounds of 1 s, 1-minute load 2.34 at the start and 3.32 at the end. Before → after, range over the rounds:operation before (µs) after (µs) Abs244–258 70–75 Neg243–249 72–75 Square243–247 72–80 Floor432–457 73–81 Ceil437–464 72–81 Round(the old one was half away from zero)325–340 71–80 Roundwas measured in a second run of the same kind (four interleaved rounds, load 2.25 to 2.42), v0.42.0 against this change.RoundHalfAway, the control on the scalar path, takes 324–345 µs in the same run. These are times of the Go code against itself; the NumPy ratios are in the table above.Ravelof a contiguous array andRealof a real array return views, as NumPy does, instead of copying (the table’s rows were 566× slower than NumPy, which returns views). On 10⁵ float64, cfarm420, one pinned core, three interleaved rounds at load 2.1 to 2.3:Ravelwent from 83–90 µs to 93–101 ns andRealfrom 84–91 µs to 101–104 ns, 803 kB to 128 B per call. The new table measures them against NumPy at 63 ns and 97 ns against 174 ns and 168 ns (ratios 0.36 and 0.58). This is a behaviour change (see the README).Float64
MaximumandMinimumrun on the go-simd/floats v0.26.0 kernels instead of the scalar builtin loop, in every path that calls them (same shape, scalar and repeated operands, broadcast). The results are bit for bit those of the builtinmaxandminfor numbers and zeros; a NaN gives a NaN, as before, and which NaN is target-dependent (TestMaximumMinimumFloats). On 10⁵ float64 including the result allocation, cfarm420, one pinned core, a test binary of main against one of this change, four interleaved rounds of 1 s at load 3.6 to 4.0 (benchmarks/maxmin_bench_test.go):Maximumwent from 139–150 µs to 76–82 µs andMinimumfrom 141–151 µs to 76–81 µs. The allocation is unchanged, 2 per call and 803 kB. No NumPy ratio is claimed for these numbers: the new table has not been run on this change.
What remains of the allocation cause is Go’s own cost. A CPU profile of a bare
make([]float64, 100000) shows about 60% garbage-collector marking (assists
in the allocating goroutine) and about 23% zeroing. NumPy frees each result
when its last reference goes and reuses that memory warm. In Go, the way to
avoid both costs is to not allocate: use the *Into forms or a Workspace.
The run was made from the measured commit’s own code. The test binary was
cross-compiled from a clean checkout of 37c1b86, the commit was shipped to
the host as a git bundle, and the binary ran from outside the clone so that the
tree stayed clean. Done that way, the recorded commit is 37c1b86 itself.
Created on by tannevaled
Modified on by tannevaled