Attention Is All You Need
The foundational paper introducing the Transformer architecture that powers all modern LLMs
paperFoundations
Core papers and the open-weight releases they led to.
BERT: Pre-training of Deep Bidirectional Transformers
Bidirectional encoder representations that revolutionized NLP transfer learning
paperLanguage Models are Few-Shot Learners
The GPT-3 paper showing that scaling language models enables few-shot learning across tasks
paperLoRA: Low-Rank Adaptation of Large Language Models
Parameter-efficient fine-tuning that reduces trainable parameters by orders of magnitude
paperChain-of-Thought Prompting Elicits Reasoning
Prompting LLMs to think step-by-step dramatically improves reasoning performance
paperConstitutional AI: Harmlessness from AI Feedback
Training helpful, harmless, and honest AI systems using AI-generated feedback
paperLlama 2: Open Foundation and Fine-Tuned Chat Models
Meta's open-source LLM family from 7B to 70B parameters, available for research and commercial use
modelMistral 7B
Compact 7B model outperforming Llama 2 13B with grouped-query and sliding window attention
model