Nite
Nite

Go GMP Model

Nite Nite included in Note
 About 3000 words 15 minutes 
Contents

GMP is the trio of core components in the Go scheduler:

  • G (Goroutine): the task created by go func().
  • M (Machine): an OS thread.
  • P (Processor): scheduling context that manages a G queue.

M is an OS thread. The runtime creates it via newm/newosproc (the functions that create M and its underlying thread), and the OS then schedules it onto a CPU core. M is the smallest unit that executes Go code; an M without a P can only sleep or handle system calls.

G states and lifecycle

A G has a well-defined set of runtime states (the enum in runtime2.go):

  • Gidle: just allocated, not yet initialized.
  • Grunnable: queued in a run queue, waiting for an M to pick it up.
  • Grunning: executing user code.
  • Gsyscall: executing a syscall, temporarily detached from scheduling.
  • Gwaiting: parked (suspended), waiting on a channel, lock, or IO.
  • Gdead: finished, sitting in a free list for reuse.
Diagram Code 
flowchart LR
    A["Gidle"] --> B["Grunnable"]
    B --> C["Grunning"]
    C --> D["Gsyscall"]
    C --> E["Gwaiting"]
    D --> B
    E --> B
    C --> F["Gdead"]
    F --> B
flowchart LR
    A["Gidle"] --> B["Grunnable"]
    B --> C["Grunning"]
    C --> D["Gsyscall"]
    C --> E["Gwaiting"]
    D --> B
    E --> B
    C --> F["Gdead"]
    F --> B
flowchart LR
    A["Gidle"] --> B["Grunnable"]
    B --> C["Grunning"]
    C --> D["Gsyscall"]
    C --> E["Gwaiting"]
    D --> B
    E --> B
    C --> F["Gdead"]
    F --> B
flowchart LR
    A["Gidle"] --> B["Grunnable"]
    B --> C["Grunning"]
    C --> D["Gsyscall"]
    C --> E["Gwaiting"]
    D --> B
    E --> B
    C --> F["Gdead"]
    F --> B

A finished G is not destroyed; it goes into a free-G cache for reuse when a new G is created. The cache has two levels: each P keeps a local cache (pp.gFree), and there is a shared global pool (sched.gFree, lock-protected). On return, a G goes into the current P’s local cache first; on creation, a new G is taken from the local cache first, and only when the local cache is empty are 32 Gs fetched from the global pool at once. Fetching in bulk minimizes global-lock contention: the next 32 creations all hit the lock-free local cache.

Stack reuse is best-effort. A G’s stack grows dynamically and may end up far larger than the initial size. If a G’s stack size doesn’t match the standard initial stack size, the stack is freed on return; the same happens when a reused G turns out to have a mismatched stack. Only a G whose stack size matches exactly can be reused together with its stack.

A goroutine stack starts at only 2KB (stackMin), far smaller than a thread’s default stack (megabytes). That is why G is lightweight and why millions of goroutines are feasible. The stack grows on demand, up to 1GB on 64-bit platforms.

Queue structure

Each P has a local G queue, and there is one shared global G queue. The global queue is lock-protected (sched.lock, the scheduler lock); the local queue is lock-free. This is the key to scheduling performance.

Diagram Code 
flowchart TB
    GQ["Global G queue
lock-protected"]
    P1["P1"]
    P2["P2"]
    P3["P3"]
    Q1["Local queue
runnext + runq[256]"]
    Q2["Local queue"]
    Q3["Local queue"]
    M1["M1 OS thread"]
    M2["M2 OS thread"]
    M3["M3 OS thread (idle)"]
    CPU["CPU core
scheduled by OS"]
    GQ -->|take G| Q1
    GQ -->|take G| Q2
    GQ -->|take G| Q3
    Q1 --- P1
    Q2 --- P2
    Q3 --- P3
    P1 --- M1
    P2 --- M2
    P3 -. idle .-> M3
    M1 --> CPU
    M2 --> CPU
flowchart TB
    GQ["Global G queue
lock-protected"]
    P1["P1"]
    P2["P2"]
    P3["P3"]
    Q1["Local queue
runnext + runq[256]"]
    Q2["Local queue"]
    Q3["Local queue"]
    M1["M1 OS thread"]
    M2["M2 OS thread"]
    M3["M3 OS thread (idle)"]
    CPU["CPU core
scheduled by OS"]
    GQ -->|take G| Q1
    GQ -->|take G| Q2
    GQ -->|take G| Q3
    Q1 --- P1
    Q2 --- P2
    Q3 --- P3
    P1 --- M1
    P2 --- M2
    P3 -. idle .-> M3
    M1 --> CPU
    M2 --> CPU
flowchart TB
    GQ["Global G queue
lock-protected"]
    P1["P1"]
    P2["P2"]
    P3["P3"]
    Q1["Local queue
runnext + runq[256]"]
    Q2["Local queue"]
    Q3["Local queue"]
    M1["M1 OS thread"]
    M2["M2 OS thread"]
    M3["M3 OS thread (idle)"]
    CPU["CPU core
scheduled by OS"]
    GQ -->|take G| Q1
    GQ -->|take G| Q2
    GQ -->|take G| Q3
    Q1 --- P1
    Q2 --- P2
    Q3 --- P3
    P1 --- M1
    P2 --- M2
    P3 -. idle .-> M3
    M1 --> CPU
    M2 --> CPU
flowchart TB
    GQ["Global G queue<br/>lock-protected"]
    P1["P1"]
    P2["P2"]
    P3["P3"]
    Q1["Local queue<br/>runnext + runq[256]"]
    Q2["Local queue"]
    Q3["Local queue"]
    M1["M1 OS thread"]
    M2["M2 OS thread"]
    M3["M3 OS thread (idle)"]
    CPU["CPU core<br/>scheduled by OS"]
    GQ -->|take G| Q1
    GQ -->|take G| Q2
    GQ -->|take G| Q3
    Q1 --- P1
    Q2 --- P2
    Q3 --- P3
    P1 --- M1
    P2 --- M2
    P3 -. idle .-> M3
    M1 --> CPU
    M2 --> CPU

A new G goes into the current P’s local queue, most preferably into the runnext slot (the next-to-run position; each P has exactly one, and it may be empty). If runnext is empty, the new G takes it directly; if it is occupied, the old G is pushed out into the runq (local run queue) ring buffer. runq is a fixed-size array of 256, so together with runnext the P holds 257 Gs.

What goes into runnext: a newly created G (go func()), a woken G that needs to run soon (ready called with next=true, e.g. the finalizer G), and a preempted G — so it can resume immediately after the STW ends.

This is the locality principle: the new G was just created by the current G, and running it on the same P means its data is likely still in the CPU cache, maximizing hit rate when it runs immediately.

When the local queue can’t take more (runnext occupied and runq full at 256), runqputslow (the function that moves half the queue to the global queue) moves the front half (the 128 earliest Gs) together with the displaced old G (the one that was sitting in runnext, not yet run) to the global queue. Old Gs go, the new G stays: the new G must run soon, so it remains in runnext; the front-half Gs have waited the longest, and once in the global queue they can be picked up by other idle Ps, which doubles as load balancing. So the “maximum length of a P’s G queue” is 256 + 1, not unbounded.

Why move half at once instead of just the displaced G?

Question: the new G takes runnext, so why not just throw the displaced old G into the global queue instead of moving 128 Gs along with it?

The answer has two layers:

The old G needs a place to go, so runq must free up space. The displaced G can’t re-enter runnext (the new G holds it) and can only go into runq, which is already full at 256. Making room in runq is a hard requirement, so front-half Gs must move.

Moving in bulk minimizes global-lock contention. The global queue has a lock (sched.lock). If we freed just one slot each time: create a G → runq full again → free one more → take the global lock again. Goroutine creation is a hot path; taking a global lock on every creation is unacceptable. Moving 128 at once takes the lock once, leaving runq half-empty (128 free slots); many subsequent enqueues then go through the lock-free local path until the queue fills up again. Moving half also leaves headroom: runq drops from 256 to 128, so new Gs don’t immediately hit the ceiling.

Question: idle Ps will steal from a full queue anyway via work stealing — why move anything at all?

Stealing has its own cost: when several idle Ps target the same full queue, they contend, and the losers retry, burning CPU. Once half the queue is in the global pool, idle Ps can just call globrunqgetbatch and grab a batch with one lock acquisition, no fighting. But work stealing is only a fallback — a full queue means production outpaces consumption, and relying on others to steal doesn’t unblock this P’s enqueue; making room is mandatory.

So the core reasons for moving half are “no room for the old G” plus “can’t take a global lock on every creation”; load balancing is a side benefit.

Number of Ps and Ms

The number of Ps equals GOMAXPROCS (maximum processors), which defaults to the number of logical CPU cores. It can be changed via the GOMAXPROCS environment variable or runtime.GOMAXPROCS(n).

The number of Ms changes dynamically: a new M is created when there is work to do but not enough idle Ms; idle Ms park in a sleep list (sched.midle, the idle-M list) waiting to be reused, and ones idle for too long are destroyed. The default cap is 10000, adjustable via runtime/debug.SetMaxThreads (set max threads).

Each M is bound to 0 or 1 P at any moment: only an M holding a P can execute Go code. An M obtains Gs through its bound P, in this order: runnext → local queue → global queue → steal from other Ps.

Thread reuse

work stealing

When a P’s local queue is empty, the order for getting Gs is: global queue first (one batch at a time via globrunqgetbatch, sized min(128, globalLen, globalLen/GOMAXPROCS+1), where globalLen is the number of Gs currently queued in the global queue; dividing by GOMAXPROCS splits the batch fairly among all Ps, and 128 is the ceiling because the local queue can’t hold more), then netpoll (picking up Gs whose IO is ready), and only then stealing from other Ps.

When stealing, the thief takes the front half of the target P’s runq (runqgrab does n = n - n/2, counting n from head), i.e. the half that entered the queue earliest. If nothing can be stolen and the global queue is empty too, the P joins the idle list and waits. The number of Ps is fixed — Ps are never destroyed; it’s Ms that sleep or get destroyed.

There’s also a fairness guard: every 61 scheduling ticks (schedtick%61 == 0) the scheduler forces a check of the global queue and takes from it first if non-empty. This prevents two Gs endlessly spawning new Gs in a local queue and starving the Gs in the global queue.

hand off

When a G blocks, the handling differs by case:

  • Ordinary blocking (channel, lock, sleep): the G is parked (suspended until its condition is met), the M stays put and keeps executing the next G from the P’s queue. No new M is needed.
  • Syscall blocking: the M enters the syscall together with the G, and the P is released (handoffp, the hand-off function); a sleeping M is woken to take over the P, or a new one is created if none exists.

When the syscall returns, the M first tries to get a P back: with an idle P available it just resumes the G; without one, the G goes into the global queue (globrunqput) and the M sleeps (stopm) until woken again.

Load balancing

Every new G creation (newproc, the function that creates a G) calls wakep (the M-waker): if there is no spinning M right now and an idle P exists, it wakes a sleeping M or creates a new one, binds it to that P, and sends it into spinning mode to hunt for Gs.

A spinning M’s order for getting Gs: local queue (the idle P it just picked up has an empty queue anyway) → global queue → steal from other Ps.

Spinning Ms don’t burn CPU forever:

  • Their count is capped at half the number of busy Ps (2 × spinningMs < GOMAXPROCS - idlePs must hold for a new one).
  • When neither the global queue nor any P has work, the spinning M returns its P to the idle list and sleeps itself (stopm → mPark, thread park) until the next wakep wakes it.

sysmon and preemption

This is not “force-kill after timeout”. sysmon (the system monitor thread) is a dedicated thread created at runtime startup (newm runs the sysmon function), responsible for several things:

  • Preemption: at most every 10ms it checks whether a G has been running on the same P for over 10ms (forcePreemptNS, the preemption time threshold); if so, it asynchronously preempts via a SIGURG signal (a Unix signal used for preemption, Go 1.14+). The G yields the CPU, goes back to the queue, and will be scheduled again later.
  • Syscall takeover: if a P has been stuck in a syscall for over ~20us, sysmon takes the P back and hands it to another M (working with hand off).
  • Netpoll fallback: if nobody has polled network events for over 10ms, sysmon polls epoll itself and wakes ready Gs.
  • Periodic GC: if no GC has run within forcegcperiod (2 minutes), sysmon forces one.

Network IO and netpoll

Network IO (socket reads/writes) is non-blocking by default in Go, built on IO multiplexing like epoll (Linux) / kqueue (macOS). When a G’s read/write isn’t ready, netpollblock calls gopark to suspend it (waitReasonIOWait), without occupying an M. Ready events are collected by netpoll, which wakes the corresponding Gs and pushes them back into queues.

So when many goroutines do network IO at once, Ms are never blocked by IO. This is the core of GMP’s support for high-concurrency network services: no matter how many connections, the number of active threads stays around GOMAXPROCS.

Note the distinction: file IO and syscall blocking take the other path (hand off, occupying an M), while network IO goes through netpoll and occupies no M.

Scheduling flow

  1. go func(){...}(): creates a G, which goes into the current P’s runnext first, then the local queue; when the local queue is full at 256, the front half of Gs plus the displaced old G move to the global queue.
  2. The M takes a G from its bound P: runnext → local queue → global queue → steal from other Ps.
  3. G blocks: on ordinary blocking the M keeps executing the next G; on syscall blocking it hands the P off to another M.
  4. Syscall returns: the G re-enters a queue, and the M grabs a P or sleeps.
Diagram Code 
flowchart LR
    A["go func()"] --> B["create G"]
    B --> C{"P local queue full?"}
    C -- no --> D["put in runnext / local queue"]
    C -- yes --> E["front half Gs + displaced old G to global queue"]
    D --> F["M takes G from P and runs it"]
    E --> F
    F --> G{"G blocks?"}
    G -- ordinary --> H["park G, M keeps running next G"]
    G -- syscall --> I["hand off P to another M"]
    H --> F
    I --> F
flowchart LR
    A["go func()"] --> B["create G"]
    B --> C{"P local queue full?"}
    C -- no --> D["put in runnext / local queue"]
    C -- yes --> E["front half Gs + displaced old G to global queue"]
    D --> F["M takes G from P and runs it"]
    E --> F
    F --> G{"G blocks?"}
    G -- ordinary --> H["park G, M keeps running next G"]
    G -- syscall --> I["hand off P to another M"]
    H --> F
    I --> F
flowchart LR
    A["go func()"] --> B["create G"]
    B --> C{"P local queue full?"}
    C -- no --> D["put in runnext / local queue"]
    C -- yes --> E["front half Gs + displaced old G to global queue"]
    D --> F["M takes G from P and runs it"]
    E --> F
    F --> G{"G blocks?"}
    G -- ordinary --> H["park G, M keeps running next G"]
    G -- syscall --> I["hand off P to another M"]
    H --> F
    I --> F
flowchart LR
    A["go func()"] --> B["create G"]
    B --> C{"P local queue full?"}
    C -- no --> D["put in runnext / local queue"]
    C -- yes --> E["front half Gs + displaced old G to global queue"]
    D --> F["M takes G from P and runs it"]
    E --> F
    F --> G{"G blocks?"}
    G -- ordinary --> H["park G, M keeps running next G"]
    G -- syscall --> I["hand off P to another M"]
    H --> F
    I --> F

P states

A P has four states (runtime2.go):

  • Pidle: in the idle-P list, queue empty, waiting to be picked up by an M.
  • Prunning: held by an M, executing Gs.
  • Pgcstop: GC’s STW (stop the world, all threads paused) phase, all Ps stop here.
  • Pdead: destroyed when GOMAXPROCS shrinks; reusable if GOMAXPROCS grows again.

GC starts with STW: all Ms reach a safe point, Ps transition to Pgcstop, and scheduling pauses. The marking phase then proceeds concurrently and Ps resume running user code. For the full GC picture, see Go GC mechanism.

M0 and G0

M0 is the first M created at program startup (the main thread). It handles initialization (installing signal handlers, creating the sysmon thread, etc.) and never exits — in mexit (the thread-exit path) m0 is wedged in place until the process ends.

G0 is each M’s dedicated scheduling goroutine. Every M creation (allocm, the function that creates an M) also creates its g0, stored in the M’s g0 field. g0 enters no queue and occupies no P; it only runs scheduler code (schedule, stack growth, signal handling, etc.). It is independent of P and tied only to its M: when the M blocks, g0 goes with it — there’s no “where does g0 live” problem.

Nor is there any “m0 and g0 unbinding”: g0 always belongs to m0, and m0 enters the scheduling loop through g0.

Hello world execution flow

  1. Program starts; m0 and its g0 are created.
  2. g0 runs schedinit (scheduler initialization): creates GOMAXPROCS Ps, initializes the global queue.
  3. newproc creates the main goroutine and puts it in some P’s local queue.
  4. m0’s g0 enters the scheduling loop, picks up the main goroutine, and m0 switches to running it.
  5. main finishes; the program exits.

runtime/trace 

import "runtime/trace"

f, _ := os.Create("trace.out")
trace.Start(f)
defer trace.Stop()

// your code

After generating the file, run go tool trace trace.out to open the Web UI, which shows G/P/M scheduling, blocking, syscalls, GC, and other events.

Updated on 2026-08-04 
CC BY-NC-SA 4.0
Go
Back | Home

Contents

Navigate Select ESC Close
Search by Fuse.js