OpenAI ML Engineer Interview (2026): Process & Prep

The OpenAI ML engineer interview is its own loop, not the generic software engineer loop with a model question bolted on. It grades whether you can implement ML primitives from scratch, design training and inference infrastructure with real GPU-memory math, and reason about evals as engineering. If you came here looking for LRU-cache and rate-limiter rounds, read the OpenAI software engineer interview guide instead. This page covers the ML track specifically: six rounds, the real question shapes, worked NumPy code, 2026 comp, and a prep plan mapped round by round. The week-four mock-session advice uses Interview Coder, which we'll get to once.

These four titles get mashed together by every competitor guide, and the loops actually diverge. Here is the clean version.

The ML engineer owns the layer between research and product: the training jobs that won't fit on one GPU, the eval suites that catch regressions, and the serving stack that keeps latency down. The research engineer sits closer to model training loops and gets pushed harder on numerics and sometimes kernels. The AI engineer lives at the application layer; for that loop, see the AI engineer interview questions guide. The biggest practical difference from the SWE loop is round two: the ML engineer gets a from-scratch ML coding round that the SWE loop does not have.

Six rounds, four to six weeks end to end. The paid 48-hour work trial is the round nobody else writes about because no other frontier lab runs one.

The inference-serving question is sometimes folded into round three and sometimes split into its own session, depending on team load. Eval engineering has become its own gradeable round in 2026 loops rather than a sub-question of system design. For the wider process, including team matching and how offers land, read how to prepare for the full OpenAI interview process.

Same six rounds, different scorecard.

Applied ML engineers ship the serving stack and the eval infra behind ChatGPT and the API. They weight the inference-serving design and eval round heaviest. Your work trial looks like a real serving or pipeline feature with tests.

Safety ML engineers build the eval and monitoring infrastructure for the Preparedness Framework: red-team harnesses, regression detection, classifier training. The eval round goes deep, and the behavioral round digs harder into judgment under uncertainty.

Research engineering supports researchers with training infrastructure and experiment tooling. Less weight on product polish, more on training-infra design, numerics correctness, and debugging across Python, C++, and CUDA. The from-scratch coding round can get more numerically demanding here.

Thirty minutes, conversational, but the ML signal screen starts here. The recruiter is confirming you have shipped real ML systems, not just trained notebooks. They listen for scale words that check out: how many GPUs, what parallelism, how big the dataset, what broke and how you found it.

Expect a past-project deep dive. Pick one project where you owned an ML system end to end and can talk about the parts that hurt: the training run that diverged, the eval that lied, the inference cost that blew the budget. Vague ownership reads as a red flag for this track because the next four rounds assume you have built this stuff.

Skip salary anchoring. If pushed, give a range and say you will firm it up after the loop. Use the back half to ask which team the role maps to and whether the work trial leans infra or product. The answer tells you which rubric above you are walking into.

This is the round that separates the ML engineer loop from the SWE loop, and the one every competitor guide names but never shows. You get a shared editor, one ML primitive, and 60 minutes. No torch.nn, no autograd. You implement the math in NumPy and explain every step. It is collaborative live coding, so narrate your shapes, ask about edge cases, and let the interviewer push you.

The three primitives that show up most: scaled dot-product and multi-head attention, k-means, and a 2D convolution. Here is attention, the single most common ask.

import numpy as np

def softmax(x, axis=-1):
    x = x - np.max(x, axis=axis, keepdims=True)  # stability
    e = np.exp(x)
    return e / np.sum(e, axis=axis, keepdims=True)

def scaled_dot_product_attention(Q, K, V, mask=None):
    # Q: (..., seq_q, d_k), K: (..., seq_k, d_k), V: (..., seq_k, d_v)
    d_k = Q.shape[-1]
    scores = Q @ K.swapaxes(-1, -2) / np.sqrt(d_k)
    if mask is not None:
        scores = np.where(mask, scores, -1e9)
    weights = softmax(scores, axis=-1)
    return weights @ V, weights

def multi_head_attention(X, Wq, Wk, Wv, Wo, num_heads):
    seq, d_model = X.shape
    d_head = d_model // num_heads
    Q, K, V = X @ Wq, X @ Wk, X @ Wv
    # split heads: (seq, d_model) -> (num_heads, seq, d_head)
    split = lambda t: t.reshape(seq, num_heads, d_head).transpose(1, 0, 2)
    Qh, Kh, Vh = split(Q), split(K), split(V)
    out, _ = scaled_dot_product_attention(Qh, Kh, Vh)
    out = out.transpose(1, 0, 2).reshape(seq, d_model)  # merge heads
    return out @ Wo

The interviewer is watching for the softmax stability subtraction, the 1/sqrt(d_k) scale, correct mask handling, and clean head reshaping. State the complexity out loud: attention is O(seq^2 * d) time and O(seq^2) memory in the scores matrix, which is exactly the constraint the inference-serving round later asks you to fight.

In practice, the moment that decides this round is rarely the math; it is the 1e9 line. Candidates who write scores = scores * mask instead of selecting with np.where quietly zero out the masked positions before the softmax instead of pushing them to -inf, so the masked tokens still leak a small amount of attention weight. The interviewer usually says nothing and waits to see whether you catch it when you sanity-check the output, because the difference between someone who has only read about attention and someone who has debugged it is whether they reach for a tiny worked example, e.g. a length-2 sequence with a causal mask, and check that the second token gets zero weight on the first. Build that check-with-a-toy-example reflex before the round; it is the single behavior that separates the two candidates.

k-means is the warm-up version of the same skill: vectorize the assignment step, avoid a Python loop over points.

def kmeans(X, k, iters=100, seed=0):
    rng = np.random.default_rng(seed)
    centroids = X[rng.choice(len(X), k, replace=False)]
    for _ in range(iters):
        d = np.linalg.norm(X[:, None] - centroids[None], axis=2)  # (n, k)
        labels = d.argmin(axis=1)
        new = np.array([X[labels == j].mean(0) if (labels == j).any()
                        else centroids[j] for j in range(k)])
        if np.allclose(new, centroids):
            break
        centroids = new
    return centroids, labels

For conv2d, the signal is handling stride, padding, and the channel dimension without an off-by-one. A naive four-loop version is acceptable if you state that the vectorized im2col version is the production path. Get the base case correct first, then offer the optimization. Candidates who chase the vectorized version immediately and botch the indexing score worse than candidates who ship the loop and explain the speedup. For the broader theory these rounds assume, work through machine learning interview questions. For the tooling and pacing side of any live round, ace the coding interview with AI.

For most ML engineer loops, Python and NumPy carry the coding round. C++ and CUDA only get tested on performance and kernel teams, mostly inside research engineering and inference-optimization roles. If the recruiter says the team writes kernels, expect a question framed as throughput without quality loss: speed up a matmul or a fused attention kernel, reason about memory coalescing, occupancy, and warp divergence, and explain why the fused version cuts HBM traffic (the FlashAttention result: tiling attention into SRAM to slash reads and writes against high-bandwidth memory). You will rarely write full CUDA on a whiteboard. You will be asked to reason about it: where the bandwidth goes, why a kernel is memory-bound versus compute-bound, and what fusing buys you. If the role is Applied serving or eval infra, you can deprioritize CUDA entirely and spend that time on the inference-serving design.

Sixty minutes, open-ended, anchored on one prompt: "train a model that won't fit on one GPU." This is where the ML engineer loop diverges hardest from SWE system design, and where competitor guides that only cover OpenAI's API-platform design leave you blind.

Start by separating the three axes of parallelism, because conflating them is the most common failure:

Real frontier training combines all three (3D parallelism); the tensor-parallel half of that mix traces back to Megatron-LM, which sharded transformer layers across GPUs to train multi-billion-parameter models. Then bring up sharding the optimizer state. ZeRO-style sharding (the optimizer states, gradients, and parameters partitioned across data-parallel ranks) is the move when the optimizer state, not the weights, is what blows memory. Do the math out loud: a model with P parameters in mixed precision needs roughly 2 bytes for fp16 weights, 2 for the gradient, and 12 for the Adam states (fp32 master weights plus two moments), so about 16 bytes per parameter before activations. That fp16-weights-plus-fp32-master-copy split is the mixed-precision training recipe, and it is why the optimizer state dwarfs the raw weights. A 70B model is therefore over 1TB of optimizer-related state, which is why sharding is not optional.

Close with the operational half that separates senior answers: fault tolerance and checkpointing. Training runs for weeks on thousands of GPUs; a node will die. Talk about asynchronous checkpointing to avoid stalling the run, checkpoint frequency as a tradeoff between recovery cost and write overhead, and elastic restart. Then data and model versioning: which data shard mix produced this checkpoint, and can you reproduce a run. A clean answer to "design a distributed training platform" names the parallelism strategy, the sharding scheme, the checkpoint and restart story, and the versioning, with memory numbers attached. For the general muscle behind this round, use system design interview preparation.

The other half of ML system design, sometimes its own session: "serve this model with low latency at high throughput." The single most important concept is the KV cache. During generation, you cache the keys and values for every past token so each new token is O(seq) instead of O(seq^2). That cache is large and grows with sequence length, so it usually dominates GPU memory at serving time, not the weights.

Do the GPU-memory math when asked. KV cache size is roughly 2 (K and V) * num_layers * num_heads * head_dim * seq_len * batch * bytes_per_value. For a large model with long context and a real batch, that runs into tens of gigabytes, which is why an H100's 80GB fills fast and why paged KV cache and prefix sharing matter.

Then continuous batching: instead of waiting for a full static batch, you add and evict requests from the running batch every step so finished sequences free their slots immediately. It is the single biggest GPU-utilization win for variable-length generation. Round out the design with the latency-versus-throughput tradeoff (bigger batches raise throughput and tail latency), streaming via server-sent events with the failure mode when a worker dies mid-stream, and multi-tenant rate limiting so one org cannot starve the rest. Anchor on numbers: time-to-first-token budget, tokens per second per request, cost per million output tokens.

In 2026 loops, eval engineering is its own gradeable round, weighted heavily, because at OpenAI a model is only as shippable as the evals that gate it. Treat this like a system design round where the system is the eval pipeline.

Hit these pieces. Golden sets: curated, versioned, leak-checked test cases that define correct behavior; talk about how you keep them from going stale and from leaking into training. LLM-as-judge: using a model to grade open-ended outputs, plus its failure modes (position bias, verbosity bias, self-preference) and how you calibrate the judge against human labels. Offline versus online evals: offline on a fixed set before deploy, online via A/B and live monitoring after. The part that wins this round is regression detection: how you catch a quality drop before it reaches production, including per-slice metrics so an aggregate that looks flat does not hide a regression on a critical subgroup, and statistical significance so you do not chase noise. Familiarity with RLHF and RL matters more on research-adjacent and safety teams, where evals feed reward models and the line between eval and training blurs. The retrieval and eval overlap shows up directly in RAG and eval interview questions; review it before this round.

You sign an NDA, get a brief, and have 48 hours. OpenAI pays a flat rate for the time. For an ML-infra candidate the project is not a product feature; it is a representative infra task: a training or data pipeline component, an eval harness, a serving optimization, or a benchmark with a writeup. A product candidate might add an API endpoint. You might be asked to profile a slow training step, implement a caching layer for an eval suite, or add metrics and a regression check to a pipeline.

Reviewers grade four things: correctness, tests (an infra task with no tests fails regardless of how clever the code is), a design doc that explains your tradeoffs and what you would do with more time, and how you handled the deliberately under-specified parts of the brief. The 48 hours include sleep. Block the timeline: a few hours on understanding plus a design-doc draft, the bulk on implementation and tests, several hours on the writeup and polish, and real buffer. A working, well-tested solution with a clear README beats a clever one with no docs.

Forty-five minutes, STAR-format project stories plus a real block on mission alignment. For ML engineers the ethical-judgment stories are specific: a time you caught an eval that was lying, a model behavior you escalated, a launch you slowed because a safety or quality metric looked off. Have two of these ready as 90-second stories.

Do the reading. Form one concrete opinion on the OpenAI Charter and one on the Preparedness Framework's risk categories. The interviewer wants to see you have reasoned about deployment risk, not that you can recite a mission statement. The "why OpenAI" answers that fail are the comp answers and the generic ones: "I want to work on AGI" signals you would say the same thing to every lab. A take like "the Preparedness Framework's capability thresholds create a real tension with shipping speed, and here is how I would weigh it" lands. Pushback grounded in understanding beats enthusiasm. For structuring the project-story half, use tell me about yourself.

ML engineer comp tracks the software engineer bands, with PPU (Profit Participation Units) driving most of the upside. As of early 2026, triangulating Levels.fyi self-reports with current recruiter conversations:

PPU appreciation has been the swing factor; engineers who joined before the latest funding rounds have seen large upside, so total comp depends heavily on PPU valuation rather than base. The default is hybrid in San Francisco, three days in office, with most new hires relocating to SF. Full remote is rare and case-by-case. For how leveling compares across companies, see engineering levels and leveling.

Four weeks is enough if you map prep to the actual rounds instead of grinding generic LeetCode.

Week 1: from-scratch ML coding. Implement attention (scaled dot-product, then multi-head), k-means, and conv2d in NumPy, three times each, timed. Add backprop for a small MLP and a layernorm if you have time. Target: any of the three primitives correct in under 30 minutes with the complexity stated out loud. Drill on machine learning interview questions so the theory behind each is automatic.

Week 2: training and inference system design. Two hours on 3D parallelism and ZeRO sharding with the 16-bytes-per-parameter math memorized. Two hours on the KV cache, continuous batching, and GPU-memory sizing. Three hours sketching both designs end to end with numbers attached. Then a timed mock with someone who will call out hand-waving, leaning on system design interview preparation.

Week 3: evals plus behavioral and mission reading. Build a tiny eval harness with a golden set and an LLM-as-judge so you can speak from having done it. Write three project stories down to 90 seconds. Read the Charter and the Preparedness Framework and form one specific opinion on each. This block matters more than any single coding hour.

Week 4: mocks plus work-trial simulation. Run timed mocks for the coding and both design rounds, plus a behavioral mock where someone pushes on your mission answers. Interview Coder runs the timed coding-mock side; its answers run on Claude Sonnet 4.6. Then simulate the work trial: take an unfamiliar repo, give yourself an under-specified infra brief, and ship it with tests and a design doc in a fixed window. For the foundations under all of this, the software engineer interview prep guide and the full OpenAI interview process walkthrough fill the gaps.

Not harder, different. The from-scratch ML coding round and the training-infra design round demand ML depth the SWE loop never tests. If you have shipped ML systems, the ML loop is more natural; if you are a pure backend engineer, the SWE loop is the easier door.

No. A PhD helps for research engineering and pure research roles, but ML engineering rewards shipped systems: training infra, eval pipelines, serving stacks. Strong applied ML experience without a PhD clears the bar regularly.

No. LeetCode does not touch attention from scratch, parallelism strategy, KV-cache math, or eval design, which is most of this loop. Use LeetCode only for general coding fluency, then spend your real time on the ML-specific rounds.

For most ML engineer roles, almost none; Python and NumPy carry the coding round. CUDA only gets tested on performance and kernel teams, mostly in research engineering and inference optimization. Confirm with your recruiter which team you are interviewing for.

Four to six weeks end to end. The paid 48-hour work trial plus scheduling adds one to two weeks over a standard loop, and team matching happens after the final round.

The OpenAI ML engineer interview is the ML track, not the SWE loop. The four things that decide it: a from-scratch NumPy coding round (attention, k-means, conv2d) the SWE loop never asks, an ML system design split into training infrastructure (parallelism, ZeRO sharding, checkpointing) and inference serving (KV cache, continuous batching, GPU-memory math), eval engineering as its own gradeable round, and a paid infra work trial. Map your four weeks to those rounds, not to LeetCode. Run timed mocks where it counts.

Interview Coder is a desktop app with 20+ stealth features and 100K+ users; its coding answers run on Claude Sonnet 4.6, Anthropic's latest Sonnet, and it covers the live ML-coding and design rounds for week-four reps. Free is $0, Monthly Pro is $299, and Lifetime Pro is $799 one-time. Full disclosure: this guide is published by Interview Coder. Start your prep at Interview Coder.