Recurrent Learning Weakens Rhythmicity

Abstract

We compared fixed and trainable reciprocal PING conductances across four initialization conditions, with three MNIST training seeds per condition. Trainable recurrence had mean reference-image contrast 0.05 after epoch 1 and ended at 0.018–0.111, below the fixed-recurrence control. E→I pruning accompanied increased E firing, but the comparison does not isolate pruning as the cause or establish equivalent accuracy. The single-image rhythmicity diagnostic does not show that recurrent learning must always destroy gamma.

Results

E→I pruning accompanies low contrast

In all three networks in each condition, training from standard and 10%-standard recurrence drove most E→I weights to zero, while most I→E weights remained positive and their mean across all entries grew. Both conditions ended with lower reference-image contrast and higher E rates than frozen recurrence. Zero-initialized recurrence remained zero (Fig. 1). This pattern across three training replicates per condition remains descriptive: it does not isolate pruning as the cause of reduced contrast or establish equivalent accuracy.

Rhythmicity starts low

The unsmoothed trainable contrast averaged 0.05 after epoch 1 and ended at 0.018–0.111. Because these histories have no epoch-0 observation, they cannot resolve the intervening transition (Fig. 2).

The first digit before and after training

We replayed the first official-test image, a digit 7, through each condition at initialization and epoch 50 (Fig. 3). The sampled E cells fire sparsely at initialization. At epoch 50, the frozen-recurrence example shows repeated E/I volleys, whereas the trainable examples show denser, less visibly synchronous E activity. These seed-42 examples use identical input spikes across all eight panels. They illustrate individual responses; they do not estimate variability across images or training seeds. The initial feedforward weights are reconstructed from the recorded training initializer, because no epoch-0 checkpoint was retained.

Methods

We reused networks from the exp022 — MNIST (legacy) and reanalysed recorded observations for Figures 1–2. Figure 3 adds matched CPU diagnostic simulations of reconstructed initialization and the retained final checkpoints. No new training was performed.

1. Compare recurrent trainability. Twelve conductance-based leaky-integrate-and-fire classifiers had 784 Poisson input channels, 1,024 excitatory (E), 256 inhibitory (I) and 10 output neurons. With three training replicates per condition, we compared frozen canonical recurrence with trainable canonical, zero and 10%-canonical E→I/I→E conductances; E→E and I→I coupling stayed zero. Canonical initializer means were $\frac{1}{1024}$ and $\frac{2}{256}$, respectively, with standard deviations one tenth of each mean and negative draws clamped to zero.

2. Train on a held-out split. The 7,000-image subset contained 6,300 optimizer-training and 700 validation images from the official MNIST training partition. Input and readout weights trained for 50 epochs with AdamW, learning rate $4 \times 10^{- 4}$, zero weight decay, batch size 256, surrogate slope 1, voltage-gradient damping 1,000, gradient-norm clipping at 1 and no firing-rate penalty. Class scores were mean pre-reset output voltages; each 200 ms presentation used 0.1 ms steps and pixel-dependent input rates up to 25 Hz.

3. Constrain conductance signs. Trainable recurrent magnitudes were projected onto the non-negative cone after each optimizer step. Inhibition arose through $g_{I} ( E_{I} - V_{m} )$, where $g_{I}$ is inhibitory conductance, $E_{I} = - 80$ mV its reversal potential, and $V_{m}$ membrane voltage: a positive I→E weight need not become negative to inhibit. Input zeros remained trainable and could regrow.

4. Evaluate final networks. All endpoint tests and weight comparisons used epoch 50, not validation-selected weights. Accuracy and whole-population mean E/I rates used the same 1000 official-test images per network; per-epoch validation metrics averaged three fixed Poisson encoding draws. For endpoint spectra, demeaned nonconstant E-population traces received full-trial Welch density estimation; the mean spectrum’s largest bin within 5–150 Hz defined the selected peak, without interpolation.

5. Measure temporal contrast. After each epoch, the same fixed reference digit’s Poisson spike realization elicited a diagnostic response. E-population counts were binned at 1 ms; their autocorrelation was normalized by lag overlap and squared mean count, over 0–100 ms, then smoothed with weights $( \frac{1}{4} , \frac{1}{2} , \frac{1}{4} )$ after replacing the zero-lag entry by its neighbour:

   $$R_{\text{contrast}} = \frac{A_{\text{lobe}} - A_{\text{trough}}}{A_{\text{lobe}} + A_{\text{trough}}} \tag{1}$$

   Here $R_{\text{contrast}}$ is dimensionless contrast, $A_{\text{trough}}$ the first local trough from lag 2 ms onward, and $A_{\text{lobe}}$ the preceding positive-lag maximum of the smoothed autocorrelogram. We reused the recorded scalar; it is neither a test-population rhythm estimate nor a calibrated probability of PING.

6. Summarize recurrent weights. We reanalysed the initial and epoch-50 E→I and I→E matrices. For each direction and condition, we calculated the strictly positive fraction and arithmetic mean including zeros, both per network and over the three pooled matrices. Equal matrix sizes gave each network equal weight. Separately at each time point, we calculated SEM as the sample standard deviation of the three network summaries divided by $\sqrt{3}$. The independently trained network was the unit of replication; jointly trained weights were not additional replicates. These descriptive summaries did not test the significance of training-induced changes.

7. Expose retained training evidence. We displayed retained validation, activity and temporal-contrast measurements with their distinct training-replicate and illustrative-probe roles.

8. Replay a matched before/after example. For seed 42, we reconstructed the input and readout matrices using the recorded CPU initialization seed, distribution, sparsity mask and draw order. Reconstructed recurrent matrices agreed with the retained initial matrices within four float32 machine epsilons of relative error; we then used the retained matrices themselves. Each epoch-50 replay used its authenticated final checkpoint. We encoded official-test image 0 once with CPU Poisson encoder seed 42 and passed the same spike tensor to all four conditions at both time points. SNNLab’s GraphExecutor reset state on every presentation. The compute run retains the image, encoded input, full E/I recordings, native parameter snapshots and explicit endpoint ancestry. This reconstructed diagnostic does not add an epoch-0 observation to the historical training curves.