Gated DeltaNet-2 improves linear attention with separate memory controls

Original: Gated DeltaNet has been one of my favorite "hybrid attention" newcomers in the good old transformer stack.

Source: magazine.sebastianraschka.com ↗