Skip to content

Interview prep · GPU infrastructure

The GPU infrastructure interview, taken apart.

What an Oracle Principal Core Infrastructure loop actually tests, the six troubleshooting questions that decide it, and the reasoning path a principal engineer is expected to walk out loud.

Nobody passes this loop by knowing more acronyms. You pass it by narrowing a problem under pressure, in front of someone who will keep asking why.

Download the PDF Free, and no signup.

1. Where the loop actually spends its time

Write down every phone-screen question you remember, especially the ones you could not answer cleanly. Loop questions are often similar or identical to the screen. That one exercise is worth more than any amount of fresh reading.

Area Share What good preparation looks like
GPU architecture and troubleshooting 25% H100/H200/B200, NVLink/NVSwitch, PCIe, NUMA, Xid/ECC, DCGM, node validation
Kubernetes and Slurm 20% GPU scheduling, device plugin, node states, pending jobs, GRES, drain and return-to-service
Networking and RDMA 15% RoCE/InfiniBand, GPUDirect RDMA, topology, congestion, drops, MTU, NIC health
Linux and node troubleshooting 15% CPU, memory, OOM, I/O wait, systemd, dmesg, cgroups, NUMA
Resume deep dives 15% Scope, architecture, metrics, tradeoffs, failures, and your personal contribution
Behavioural and OCI values 10% Ownership, ambiguity, conflict, failure, influence, learning, measurable outcomes

2. The shape of every good answer

Six steps, in this order, whatever the domain. Say them out loud as you go. An interviewer who can hear the structure stops testing whether you have one.

  1. 01 Clarify scope and symptoms.
  2. 02 State the most likely failure domains.
  3. 03 Start with the highest-signal, lowest-cost checks.
  4. 04 Use telemetry or commands to prove or disprove each hypothesis.
  5. 05 Mitigate blast radius if production is affected.
  6. 06 Find root cause, validate the fix, and explain prevention.

3. The six domains

For each one: what you need to know, the question you will actually be asked, and the order in which you are expected to eliminate things. The path is the part worth memorising. It is not an answer, it is a route.

GPU node

Know. H100 vs H200 vs B200; SXM vs PCIe; HBM; NVLink/NVSwitch; MIG; ECC/Xid; drivers and CUDA; DCGM; PCIe/NUMA topology.

One GPU in an 8-GPU node is 40% slower. Walk me through troubleshooting.

Good path. Application → utilisation and memory → thermals and clocks → ECC/Xid → NVLink → PCIe/NUMA → network → hardware.

Cluster and distributed

Know. Single-node vs multi-node bottlenecks; NCCL; stragglers; topology; scheduler placement; network and storage coupling.

Training is fast on one node but much slower on 32 nodes. Why?

Good path. GPU compute → NCCL → NVLink → NIC/RDMA → fabric congestion → topology → stragglers → storage and data pipeline.

Slurm

Know. slurmctld/slurmd, partitions, GRES, node states, draining, job states, scheduling, health checks.

A GPU job is stuck Pending. How do you determine why?

Good path. Reason field → requested resources → partition constraints → GRES → node state → reservations and limits → scheduler logs.

Kubernetes

Know. NVIDIA device plugin and GPU Operator, allocatable GPU resources, kubelet, taints and tolerations, affinity, runtime and CNI.

A pod requests a GPU but stays Pending even though GPUs appear available.

Good path. Describe pod → scheduler reason → allocatable GPU → device plugin → node health → taints and affinity → fragmentation and driver state.

Network

Know. TCP/IP, DNS, MTU, bonding, RDMA, RoCE/InfiniBand, GPUDirect RDMA, congestion, packet drops, topology.

Nodes communicate successfully, but distributed training performance is poor.

Good path. Separate connectivity from performance: throughput and latency → errors and drops → RDMA counters → MTU → congestion → topology and oversubscription.

Linux and storage

Know. CPU load, memory and OOM, I/O wait, NUMA, kernel logs, filesystem; IOPS vs throughput, metadata, NVMe cache, parallel storage.

GPU utilisation is only 30%. What do you inspect?

Good path. CPU and data loader → memory and NUMA → storage → network → synchronisation → application pipeline, before blaming the GPU.

4. The resume deep dive

Any strong bullet on your resume is a ten-minute technical interrogation waiting to happen. Be precise about ownership, and never hide behind "we".

  • What was the exact scale: nodes, GPUs, workloads, users?
  • What was the architecture, and why was it designed that way?
  • What problem or bottleneck were you solving?
  • What metrics proved both the problem and the improvement?
  • What did you personally design, implement, decide or troubleshoot?
  • What alternatives did you consider, and what tradeoffs did you make?
  • What failed or surprised you?
  • What would you do differently today?

If you claim H100 or B200 clusters, Grafana node validation, Slurm failures or inference optimisation, be ready to explain the actual telemetry, the topology, the root cause and the before-and-after number.

5. The three behavioural questions

Q1. Tell me about a time you had a major production issue or infrastructure failure. How did you handle it?

  • They are testing. Ownership under pressure, troubleshooting discipline, communication, prioritisation and prevention.
  • Answer shape. Situation → impact → what you did → hypotheses and evidence → mitigation → root cause → permanent fix → measurable result → learning.
  • Expect follow-ups. What was the root cause? What did you do first? How did you know? What would you do differently? How did you prevent recurrence?

Avoid "the team investigated and fixed it". Show your personal decisions and your technical reasoning.

Q2. Tell me about a time you had to make a technical decision without all the information you needed.

  • They are testing. Judgment under ambiguity, risk management, learning speed, and knowing when to escalate.
  • Answer shape. Unknowns → constraints → separate knowns from assumptions → identify the highest-risk assumption → proof of concept → rollback criteria → decision → result → learning.
  • Expect follow-ups. What assumption was wrong? What if the decision had failed? Who disagreed? What evidence changed your mind?

The principal signal here is distinguishing reversible from irreversible decisions, and showing how you reduced uncertainty before committing.

Q3. Tell me about a technical decision you made that turned out to be wrong.

  • They are testing. Self-awareness, accountability, adaptability and learning.
  • Answer shape. Decision → why it looked reasonable then → evidence you were wrong → how quickly you changed course → impact → what you changed in your decision process.
  • Expect follow-ups. Pick a real mistake with meaningful consequences, not a disguised strength.

Avoid blaming incomplete requirements, another team or a junior engineer. Own the decision and show the improvement in judgment.

6. STAR, compressed

Situation Task Action Result
Set context and stakes quickly. What were you accountable for? Spend most of your time here. Use "I". Explain why. Quantify the impact, and say what you learned.

7. Sixty seconds before you join

  • Do I remember every phone-screen question, and my weak answers?
  • Can I explain an 8-GPU node and a multi-node training path end to end?
  • Can I troubleshoot a bad node, a Pending Slurm job, and a Pending GPU pod?
  • Can I explain RDMA/RoCE/InfiniBand at a practical troubleshooting level?
  • Can I defend every GPU, Slurm, Grafana and Kubernetes claim on my resume?
  • Do I have six to eight STAR stories with measurable outcomes?
  • Am I asking clarifying questions before jumping to a solution?
  • Am I saying what I personally did, not only what the team did?

Do not try to sound encyclopedic. Sound systematic. A principal engineer makes the interviewer confident that when the answer is not obvious, they know how to narrow the problem, use evidence, manage risk, and drive it to closure.

Where to go from here

This sheet tells you what you will be asked. It cannot make you rehearse, and the part engineers leave until last is never the GPU question.

If you would rather work through your own answers with me directly, coaching is here.

More written guides at askgurpreet.com/system-design. Want your own answers scored? Free, five minutes.