Glossary › GRPO

GRPO (Group Relative Policy Optimization)

A reinforcement learning algorithm for post-training that samples a group of responses per prompt and uses each response's reward relative to the group's mean as its advantage. Drops the value model that PPO needs, which makes it cheaper and more stable. The workhorse behind most reasoning models.

← Excessive Agency · Guardian Pattern → · All terms

Where GRPO shows up

All matching posts

Agent EngineeringADVANCED8 min read

Reasoning Models, Post-Training, and Test-Time Compute

Where reasoning models come from, what they cost, and when to pay for them. The SFT to DPO to GRPO pipeline, reinforcement learning with verifiable rewards, test-time compute as a dial, and a method for deciding which of your requests deserve a thinking model.

#reasoning-models#post-training#rl#fine-tuning#llm#model-routing

Agent EngineeringADVANCED5 min read

GRPO, DPO, and RLVR: Post-Training Explained Without the Notation

Three acronyms show up in every reasoning-model paper. Here is what each one does, why GRPO replaced PPO for most teams, what a verifiable reward is and why it is the whole constraint, and short runnable code for each idea.

#post-training#rl#reasoning-models#fine-tuning#llm