Operate · GPU Fleet

One failed GPU stops a whole job. Know which, whose and what it costs.

Telemetry per GPU in the shape of the NVIDIA DCGM exporter (and Intel Gaudi / XPU Manager), joined to the job, the customer and the building. See the job with the biggest blast radius, the slowest job, GPU time lost per hour and the work to redo since the last checkpoint.

Worked example: one failed GPU in a training job

152GPUs stopped by one failed GPU
$625/hGPU time lost on that job

Who opens it

When a job slows and nobody knows why.

GPU Fleet joins the GPU to the job, the customer and the building, so the engineer, the finance lead and the capacity manager read one story.

GPU platform engineer asksWhich job did this Xid stop, and for which customer?

The Xid, the node, the job it stalled and the customer who owns it, on one card.

Finance asksHow much GPU time are we losing per hour?

GPU time lost per hour on every hurt job, and the work to redo since the last checkpoint.

Capacity manager asksWhere should the next 256-GPU job go?

Pods ranked by free healthy GPUs, rack headroom, cooling and the marginal power cost per GPU-hour.

On the screen

What the screen holds.

The job with the biggest blast radius, the slowest job, per-GPU telemetry and where the next job should go.

Also on the screen

Network rails

InfiniBand, Ethernet front-end and OOB planes, with ports and errors per rail.

Cooling and power cost

Power per installed and per sold GPU-hour against the price, energy per job and per tenant, and the cost of idle allocations.

See your own estate in it.

Bring the fault your team keeps dealing with. We walk it end to end on a demo estate shaped like yours, then agree what a one-site pilot should prove.