# Go GMP Model


GMP is the trio of core components in the Go scheduler:

- G (Goroutine): the task created by `go func()`.
- M (Machine): an OS thread.
- P (Processor): scheduling context that manages a G queue.

M is an OS thread. The runtime creates it via `newm`/`newosproc` (the functions that create M and its underlying thread), and the OS then schedules it onto a CPU core. M is the smallest unit that executes Go code; an M without a P cannot execute Go code: it sleeps until woken to pick up a P.

## One-Paragraph Recap

The Go scheduler uses three components: G is a goroutine, with stacks starting at 2KB and growing dynamically, recycled through a two-level free cache after exit; M is an OS thread, the smallest unit that executes Go code; P is the scheduling context, its count equals GOMAXPROCS, and it owns a local G queue. New Gs go to the current P's `runnext` first, then the local `runq` (256 slots); when the local queue is full, half of it (128) is moved to the global queue together with the displaced old G, batching the move to avoid taking the global lock repeatedly. An M picks Gs in this order: `runnext` → local queue → global queue → stealing half of another P's queue, and every 61st scheduling tick it takes one from the global queue to prevent starvation. Ordinary blocking only parks the G while the M continues with the next one; blocking syscalls hand off the P to another M. sysmon handles 10ms preemption (SIGURG), reclaims a P stuck in a syscall for over 20us, falls back to netpoll, and forces a GC every 2 minutes. Network IO goes through netpoll without occupying an M, so active threads stay around GOMAXPROCS even with massive connection counts. M0 is the main thread that never exits; g0 is each M's dedicated scheduling goroutine.

## G states and lifecycle

A G has a well-defined set of runtime states (the enum in `runtime2.go`):

- Gidle: just allocated, not yet initialized.
- Grunnable: queued in a run queue, waiting for an M to pick it up.
- Grunning: executing user code.
- Gsyscall: executing a syscall, temporarily detached from scheduling.
- Gwaiting: parked (suspended), waiting on a channel, lock, or IO.
- Gdead: finished, sitting in a free list for reuse.

```mermaid
flowchart LR
    A["Gidle"] --> B["Grunnable"]
    B --> C["Grunning"]
    C --> D["Gsyscall"]
    C --> E["Gwaiting"]
    D --> B
    E --> B
    C --> F["Gdead"]
    F --> B
```

A finished G is not destroyed; it goes into a free-G cache for reuse when a new G is created. The cache has two levels: each P keeps a local cache (`pp.gFree`), and there is a shared global pool (`sched.gFree`, lock-protected). On return, a G goes into the current P's local cache first; on creation, a new G is taken from the local cache first, and only when the local cache is empty are **32 Gs fetched from the global pool** at once. Fetching in bulk minimizes global-lock contention: the next 32 creations all hit the lock-free local cache.

Stack reuse is best-effort. A G's stack grows dynamically and may end up far larger than the initial size. If a G's stack size doesn't match the standard initial stack size, the stack is freed on return; the same happens when a reused G turns out to have a mismatched stack. Only a G whose stack size matches exactly can be reused together with its stack.

A goroutine stack starts at only 2KB (`stackMin`), far smaller than a thread's default stack (megabytes). That is why G is lightweight and why millions of goroutines are feasible. The stack grows on demand, up to 1GB on 64-bit platforms.

## Queue structure

Each P has a local G queue, and there is one shared global G queue. The global queue is lock-protected (`sched.lock`, the scheduler lock); the local queue is lock-free. This is the key to scheduling performance.

```mermaid
flowchart TB
    GQ["Global G queue<br/>lock-protected"]
    P1["P1"]
    P2["P2"]
    P3["P3"]
    Q1["Local queue<br/>runnext + runq[256]"]
    Q2["Local queue"]
    Q3["Local queue"]
    M1["M1 OS thread"]
    M2["M2 OS thread"]
    M3["M3 OS thread (idle)"]
    CPU["CPU core<br/>scheduled by OS"]
    GQ -->|take G| Q1
    GQ -->|take G| Q2
    GQ -->|take G| Q3
    Q1 --- P1
    Q2 --- P2
    Q3 --- P3
    P1 --- M1
    P2 --- M2
    P3 -. idle .-> M3
    M1 --> CPU
    M2 --> CPU
```

A new G goes into the current P's local queue, most preferably into the `runnext` slot (the next-to-run position; each P has exactly one, and it may be empty). If `runnext` is empty, the new G takes it directly; if it is occupied, the old G is pushed out into the `runq` (local run queue) ring buffer. `runq` is a fixed-size array of 256, so together with `runnext` the P holds 257 Gs.

What goes into `runnext`: a newly created G (`go func()`), a woken G that needs to run soon (`ready` called with next=true, e.g. the finalizer G), and a preempted G — so it can resume immediately after the STW ends.

This is the locality principle: the new G was just created by the current G, and running it on the same P means its data is likely still in the CPU cache, maximizing hit rate when it runs immediately.

When the local queue can't take more (runnext occupied **and** runq full at 256), `runqputslow` (the function that moves half the queue to the global queue) moves **the front half** (the 128 earliest Gs) together with the displaced old G (the one that was sitting in `runnext`, not yet run) to the global queue. Old Gs go, the new G stays: the new G must run soon, so it remains in runnext; the front-half Gs have waited the longest, and once in the global queue they can be picked up by other idle Ps, which doubles as load balancing. So the "maximum length of a P's G queue" is 256 + 1, not unbounded.

**the new G takes runnext, so why not just throw the displaced old G into the global queue instead of moving 128 Gs along with it?**

Moving only the displaced G to the global queue doesn't work: the displaced G can't re-enter runnext (the new G holds it) and can only go into runq, which is already full at 256. Making room in runq is a hard requirement, so front-half Gs must move.

Moving in bulk minimizes global-lock contention. The global queue has a lock (`sched.lock`). If we freed just one slot each time: create a G → runq full again → free one more → take the global lock again. Goroutine creation is a hot path; taking a global lock on every creation is unacceptable. Moving 128 at once takes the lock once, leaving runq half-empty (128 free slots); many subsequent enqueues then go through the lock-free local path until the queue fills up again. Moving half also leaves headroom: runq drops from 256 to 128, so new Gs don't immediately hit the ceiling.

**idle Ps will steal from a full queue anyway via work stealing — why move anything at all?**

Stealing has its own cost: when several idle Ps target the same full queue, they contend, and the losers retry, burning CPU. Once half the queue is in the global pool, idle Ps can just call `globrunqgetbatch` and grab a batch with one lock acquisition, no fighting. But work stealing is only a fallback — a full queue means production outpaces consumption, and relying on others to steal doesn't unblock this P's enqueue; making room is mandatory.

So the core reasons for moving half are "no room for the old G" plus "can't take a global lock on every creation"; load balancing is a side benefit.

## Number of Ps and Ms

The number of Ps equals GOMAXPROCS (maximum processors), which defaults to the number of logical CPU cores. It can be changed via the `GOMAXPROCS` environment variable or `runtime.GOMAXPROCS(n)`.

The number of Ms changes dynamically: a new M is created when there is work to do but not enough idle Ms; idle Ms park in a sleep list (`sched.midle`, the idle-M list) waiting to be reused, and ones idle for too long are destroyed. The default cap is 10000, adjustable via `runtime/debug.SetMaxThreads` (set max threads).

Each M is bound to 0 or 1 P at any moment: only an M holding a P can execute Go code. An M obtains Gs through its bound P, in this order: `runnext` → local queue → global queue → steal from other Ps.

## Thread reuse

### work stealing

When a P's local queue is empty, the order for getting Gs is: global queue first (one batch at a time via `globrunqgetbatch`, sized `min(128, globalLen, globalLen/GOMAXPROCS+1)`, where globalLen is the number of Gs currently queued in the global queue; dividing by GOMAXPROCS splits the batch fairly among all Ps, and 128 is the ceiling because the local queue can't hold more), then netpoll (picking up Gs whose IO is ready), and only then stealing from other Ps.

When stealing, the thief takes **the front half** of the target P's `runq` (`runqgrab` does `n = n - n/2`, counting n from head), i.e. the half that entered the queue earliest. If nothing can be stolen and the global queue is empty too, the P joins the idle list and waits. With GOMAXPROCS unchanged, the number of Ps is fixed — Ps are never destroyed; it's Ms that sleep or get destroyed.

There's also a fairness guard: every 61 scheduling ticks (`schedtick%61 == 0`), if the global queue is non-empty, the scheduler skips the local queue and takes one G straight from the global queue to run. This prevents two Gs endlessly spawning new Gs in a local queue and starving the Gs in the global queue.

### hand off

When a G blocks, the handling differs by case:

- Ordinary blocking (channel, lock, sleep): the G is parked (suspended until its condition is met), the M stays put and keeps executing the next G from the P's queue. No new M is needed.
- Syscall blocking: the M enters the syscall together with the G, and the P is released (`handoffp`, the hand-off function); a sleeping M is woken to take over the P, or a new one is created if none exists.

When the syscall returns, the M first tries to get a P back: with an idle P available it just resumes the G; without one, the G goes into the global queue (`globrunqput`) and the M sleeps (`stopm`) until woken again.

## Load balancing

Every new G creation (`newproc`, the function that creates a G) calls `wakep` (the M-waker): if there is no spinning M right now and an idle P exists, it wakes a sleeping M or creates a new one, binds it to that P, and sends it into spinning mode to hunt for Gs.

A spinning M's order for getting Gs: local queue (the idle P it just picked up has an empty queue anyway) → global queue → steal from other Ps.

Spinning Ms don't burn CPU forever:

- Their count is capped at half the number of busy Ps (`2 × spinningMs < GOMAXPROCS - idlePs` must hold for a new one).
- When neither the global queue nor any P has work, the spinning M returns its P to the idle list and sleeps itself (`stopm` → `mPark`, thread park) until the next `wakep` wakes it.

## sysmon and preemption

This is not "force-kill after timeout". sysmon (the system monitor thread) is a dedicated thread created at runtime startup (`newm` runs the `sysmon` function), responsible for several things:

- **Preemption**: it checks every 20us while there is work to do, stretching to at most 10ms when idle. If the same scheduling tick (`schedtick`) has lasted over 10ms (`forcePreemptNS`, the preemption time threshold) — whether from a single long-running G or a sequence of Gs chained through runnext sharing one tick — it preempts asynchronously via a SIGURG signal (a Unix signal used for preemption, Go 1.14+). The G yields the CPU, goes back to the queue, and will be scheduled again later.
- **Syscall takeover**: if a P has been stuck in a syscall for over ~20us, sysmon takes the P back and hands it to another M (working with hand off).
- **Netpoll fallback**: if nobody has polled network events for over 10ms, sysmon polls epoll itself and wakes ready Gs.
- **Periodic GC**: if no GC has run within `forcegcperiod` (2 minutes), sysmon forces one.

## Network IO and netpoll

Network IO (socket reads/writes) is non-blocking by default in Go, built on IO multiplexing like epoll (Linux) / kqueue (macOS). When a G's read/write isn't ready, `netpollblock` calls `gopark` to suspend it (waitReasonIOWait), **without occupying an M**. Ready events are collected by netpoll, which wakes the corresponding Gs and pushes them back into queues.

So when many goroutines do network IO at once, Ms are never blocked by IO. This is the core of GMP's support for high-concurrency network services: no matter how many connections, the number of active threads stays around GOMAXPROCS.

Note the distinction: file IO and syscall blocking take the other path (hand off, occupying an M), while network IO goes through netpoll and occupies no M.

## Scheduling flow

1. `go func(){...}()`: creates a G, which goes into the current P's `runnext` first, then the local queue; when the local queue is full at 256, the front half of Gs plus the displaced old G move to the global queue.
2. The M takes a G from its bound P: `runnext` → local queue → global queue → steal from other Ps.
3. G blocks: on ordinary blocking the M keeps executing the next G; on syscall blocking it hands the P off to another M.
4. Syscall returns: the G re-enters a queue, and the M grabs a P or sleeps.

```mermaid
flowchart LR
    A["go func()"] --> B["create G"]
    B --> C{"P local queue full?"}
    C -- no --> D["put in runnext / local queue"]
    C -- yes --> E["front half Gs + displaced old G to global queue"]
    D --> F["M takes G from P and runs it"]
    E --> F
    F --> G{"G blocks?"}
    G -- ordinary --> H["park G, M keeps running next G"]
    G -- syscall --> I["hand off P to another M"]
    H --> F
    I --> F
```

## P states

A P has four states (`runtime2.go`):

- Pidle: in the idle-P list, queue empty, waiting to be picked up by an M.
- Prunning: held by an M, executing Gs.
- Pgcstop: GC's STW (stop the world, all threads paused) phase, all Ps stop here.
- Pdead: destroyed when GOMAXPROCS shrinks; reusable if GOMAXPROCS grows again.

GC starts with STW: all Ms reach a safe point, Ps transition to Pgcstop, and scheduling pauses. The marking phase then proceeds concurrently and Ps resume running user code. For the full GC picture, see [Go GC mechanism](/posts/go-gc/).

## M0 and G0

M0 is the first M created at program startup (the main thread). It handles initialization (installing signal handlers, creating the sysmon thread, etc.) and never exits — in `mexit` (the thread-exit path) m0 is wedged in place until the process ends.

G0 is each M's dedicated scheduling goroutine. Every M creation (`allocm`, the function that creates an M) also creates its g0, stored in the M's `g0` field. g0 enters no queue and occupies no P; it only runs scheduler code (schedule, stack growth, signal handling, etc.). It is independent of P and tied only to its M: when the M blocks, g0 goes with it — there's no "where does g0 live" problem.

Nor is there any "m0 and g0 unbinding": g0 always belongs to m0, and m0 enters the scheduling loop through g0.

## Hello world execution flow

1. Program starts; m0 and its g0 are created.
2. g0 runs `schedinit` (scheduler initialization): creates GOMAXPROCS Ps, initializes the global queue.
3. `newproc` creates the main goroutine and puts it in some P's local queue.
4. m0's g0 enters the scheduling loop, picks up the main goroutine, and m0 switches to running it.
5. main finishes; the program exits.

## runtime/trace

```go
import "runtime/trace"

f, _ := os.Create("trace.out")
trace.Start(f)
defer trace.Stop()

// your code
```

After generating the file, run `go tool trace trace.out` to open the Web UI, which shows G/P/M scheduling, blocking, syscalls, GC, and other events.

## Interview Questions

### What are the responsibilities of G, M, and P? Why not use only G and M?

- G stores a goroutine's execution state, including its stack, program counter, and scheduling data.
- M is an OS thread that executes instructions on a CPU.
- P holds scheduling context and manages a local run queue and runtime caches. An M needs a P to execute Go code.

With only G and M, every thread would access the global run queue directly, causing frequent lock contention during G creation and scheduling. P lets most operations stay on local queues. If an M blocks in a syscall, its P can move to another M, so scheduler capacity is not trapped with the blocked thread.

### What scheduling steps follow the creation of a goroutine by `go func()`?

The compiler turns a `go` statement into a call to `newproc`. `newproc1` first tries to obtain a G from the current P's free-G cache and allocates one only if necessary. It initializes the stack and entry point, then changes the G to Grunnable.

The new G goes to the current P's `runnext` first and then to `runq`. In the scheduling loop, an M checks `runnext`, the local queue, the global queue, and network polling, then attempts to steal work from another P.

### When does work stealing happen, and how many Gs are stolen at a time?

When the current P has no runnable G, the scheduler first checks the global queue and netpoll. It enters work stealing only when those sources have no work.

`runqsteal` uses `runqgrab` to take about half of the target P's `runq` from its head and returns one of those Gs to the current M immediately. If the target queue is empty, the scheduler may also try to steal `runnext`, but it briefly backs off to avoid moving a G between two Ps just before it runs.

### When does hand off occur? Does ordinary blocking hand off the P?

Blocking on a channel, lock, or sleep parks only the current G. The M keeps its P and can execute another G from the queue, so no hand off is needed.

When an M enters a blocking syscall with its G, `handoffp` transfers the P to another M. The runtime wakes an idle M or creates one on demand. After the syscall returns, the original M tries to acquire an idle P; if that fails, it puts the G on the global queue and sleeps.

### What does sysmon do?

sysmon runs independently without a P and monitors the runtime. It requests preemption of long-running Gs and retakes Ps from long-blocking syscalls.

If normal scheduling paths have not polled the network for a while, sysmon performs a non-blocking netpoll. It also triggers a forced GC after a long period without one and adjusts its checking interval according to timers and scheduler activity.

### How does preemptive scheduling work since Go 1.14?

When sysmon sees that a P has exceeded `forcePreemptNS` (10ms) without advancing its scheduling tick, it calls `preemptone`. This function sets the G's preemption flag and stack guard. On supported platforms, it also calls `preemptM` to send a preemption signal to the M.

The signal path redirects execution to `asyncPreempt`, allowing the G to stop at an asynchronous safe point and return to scheduling. The G is not killed, and preemption is not guaranteed to finish immediately after the request. A G inside a syscall is handled by the P-retake path instead.

### What does GOMAXPROCS mean? Does increasing it always improve performance?

GOMAXPROCS sets the number of Ps that may execute Go code concurrently. It therefore limits how many Ms can run user Go code in parallel at a given moment. It does not limit the total number of goroutines or the total number of threads the runtime may create.

A larger value may let CPU-bound workloads use more cores, but scheduler and parallel GC overhead can also rise. Setting it above the available CPU quota usually provides no further speedup. Frequent blocking syscalls can still make the number of Ms exceed GOMAXPROCS.

### How does GMP differ from a one-thread-per-task model?

In a one-thread-per-task model, the OS performs most task scheduling. Thread stacks are larger, context switches enter the kernel, and many blocking tasks can require many threads.

GMP multiplexes many Gs over a smaller set of Ms. P reduces global contention with local queues, ordinary blocking parks only the G, and network IO wakes Gs through netpoll. Threads and Ps need to separate only for cases such as blocking syscalls.


---

> Author: Nite  
> URL: https://www.nite07.com/en/posts/go-gmp/