Untied Word Embeddings Beat Tied Ones for Private AI Training

Original: Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?

Source: arxiv.org ↗