GTFormer Redefines Language Models with Linear Scaling Power!

A new preprint reveals GTFormer, a language model architecture that replaces Transformer's self-attention with a hierarchical graph neural network. Developed by Elisabeth Fink, GTFormer achieves linear time and memory scaling, demonstrating zero-shot extrapolation to contexts over four million tokens. This novel approach avoids traditional limitations, capturing long-range dependencies efficiently without positional encodings.