MNIST

Abstract

We assembled 102 trained spiking networks to recognise handwritten MNIST digits. Seven training groups compare network types, activity limits, timing, connection settings, and input rates. Their saved models and learning histories provide a reusable bank for later experiments.

Training-run overview

The retained bank groups 102 trained models into seven training families, each with initialization seeds 42–44. The table lists each family once; the model count includes all conditions and seeds within it. Dataset sizes refer to distinct images in the pool, followed by the optimizer-training and held-out validation split.

All families used the original 28 × 28 MNIST pixel intensities, scaled to [0, 1] and flattened into 784 input channels, without extracted or variable feature sets. “Variable-rate” means changing the Poisson encoding rate of those same pixels. For TR06, the readout sums output spikes over the presentation—the final cumulative spike count, not a cumulative sum of voltages. The official 10,000-image test partition was excluded from training and checkpoint selection. All families trained for 50 epochs; Training-run specification sheets gives the detailed conditions.

Results

The seven figures pair each training family’s learning history with one example of its network activity. Learning curves use validation images; each raster is a separate fixed digit-zero probe of a final-epoch model.

TR01 — Full-data baseline

Best-validation accuracy ranged from 91.51–96.04% across this family’s conditions and seeds. The accompanying raster is one final-checkpoint diagnostic, rather than an estimate of the family mean.

TR02 — Activity limits

Best-validation accuracy ranged from 60.95–91.05% across this family’s conditions and seeds. The accompanying raster is one final-checkpoint diagnostic, rather than an estimate of the family mean.

TR03 — Inhibitory decay

Best-validation accuracy ranged from 78.05–91.38% across this family’s conditions and seeds. The accompanying raster is one final-checkpoint diagnostic, rather than an estimate of the family mean.

TR04 — Timestep

Best-validation accuracy ranged from 88.67–90.33% across this family’s conditions and seeds. The accompanying raster is one final-checkpoint diagnostic, rather than an estimate of the family mean.

TR05 — Recurrent connections

Best-validation accuracy ranged from 89.10–92.24% across this family’s conditions and seeds. The accompanying raster is one final-checkpoint diagnostic, rather than an estimate of the family mean.

TR06 — Variable input rate

Best-validation accuracy ranged from 66.67–66.86% across this family’s conditions and seeds. The accompanying raster is one final-checkpoint diagnostic, rather than an estimate of the family mean.

TR07 — Input strength

Best-validation accuracy ranged from 76.43–77.38% across this family’s conditions and seeds. The accompanying raster is one final-checkpoint diagnostic, rather than an estimate of the family mean.

Methods

The completed bank contains 102 models: 90 reused models with unchanged weights and histories, plus twelve newly trained timestep models. The three existing 0.1-ms timestep models were reused after execution-equivalence checks. We analysed these histories and generated 34 final-epoch seed-42 diagnostic recordings.

1. Prepare the data. Stratified MNIST splits provided 54,000 training and 6,000 validation images for the baseline, and 6,300 and 700 for sweeps. The split seed was 42; the official test set was excluded from training and model selection.

2. Build the networks. Each network contained 1,024 excitatory neurons and ten spiking leaky integrate-and-fire outputs. Recurrent networks added feedback through 256 inhibitory neurons; feedforward controls disabled it. Input and output projections were trained, while recurrent projections were fixed except in designated conditions. Weights retained their excitatory or inhibitory sign. Hidden E/I neurons used 1.2/0.6-ms absolute refractory holds; these were already the effective durations in the reused 0.1-ms executions.

3. Vary the conditions. Seven families covered the baseline, activity ceilings, inhibitory decay, timestep, recurrent initialization and trainability, and input drive. Each condition used initialization seeds 42–44; Training-run specification sheets lists the grids and shared settings.

4. Present the images. Pixels generated Poisson spikes over nominally 200 ms, normally with a maximum-pixel rate of 25 Hz and a 0.1 ms timestep. Variable-rate training sampled uniformly from eleven rates between 0.5 and 25 Hz. Timestep conditions ranged from 0.05 to 0.6 ms; whole-step presentations lasted 199.8 ms at 0.3/0.6 ms and 200 ms otherwise. Firing rates used realised durations.

5. Calculate class scores. Most networks used mean pre-reset output voltage. Variable-rate training used spike counts and smaller initial readout weights.

   $$z_{\text{voltage}} = \frac{1}{N_{t}} \sum_{k = 1}^{N_{t}} u_{\text{out} [ k ]} , \quad z_{\text{count}} = \sum_{k = 1}^{N_{t}} s_{\text{out} [ k ]} . \tag{1}$$

   Each score applies to one output neuron and presentation. Here $k$ indexes the $N_{t}$ timesteps, $u_{\text{out} [ k ]}$ is dimensionless pre-reset output state, and $s_{\text{out} [ k ]}$ is a dimensionless binary output spike; both scores are dimensionless.

6. Train the networks. Training used 50 epochs of AdamW, learning rate 0.0004, batches of 256, zero weight decay, and gradient clipping at 1. Surrogate gradients approximated spike derivatives[1]; voltage-gradient damping was 1,000 for recurrent networks and 1 for controls. Activity-constrained conditions minimized

   $$L_{\text{total}} = L_{\text{CE}} + \lambda_{\text{rate}} {\langle {\max ( 0 , r_{E} - r_{E , \text{ceil}} )}^{2} \rangle}_{\text{batch}} . \tag{2}$$

   $L_{\text{total}}$ is total loss and $L_{\text{CE}}$ is mean cross-entropy. Each presentation’s mean excitatory firing rate $r_{E}$ and ceiling $r_{E , \text{ceil}}$ are in hertz. Brackets average the individual penalties across the minibatch; $\lambda_{\text{rate}} = 0.041$ with rates in hertz, or zero when disabled.

The selected historical bank predates the current recipe’s common voltage-gradient damping of 1,000 for COBA and PING. Its displayed settings and outcomes remain bound to that historical evidence.

7. Select models. Validation averaged three fixed Poisson encoding draws. Selection minimized mean cross-entropy, breaking ties by higher accuracy and then earlier epoch. Selected and final-epoch models were retained.

8. Measure activity and retain models. Accuracy used selected models, whereas firing rates averaged final-epoch validation measurements across images and encoding draws. Retained models and histories support subsequent experiments. Learning curves show individual validation histories; baseline summaries average three seeds. The 34 new seed-42 digit-zero recordings covered every condition, including reused networks; their rasters are individual probes, not across-seed estimates. Spectra used the realised analysis-bin duration: grouping approximately 1 ms of simulation steps gave 0.9-ms bins at the 0.3-ms timestep and 1.2-ms bins at 0.6 ms.