Sparse spiking language model matches GPT-2 with 89% fewer activations

Original: SymbolicLight V1: Spike-Gated Dual-Path Language Modeling with High Activation Sparsity and Sub-Billion-Scale Pre-Training Evidence

Source: arxiv.org ↗