GPU and HPC resource utilization optimization

AIThis description was written by a model from public signals. The facts behind it are listed under Evidence.

Problem

Datacenters and HPC clusters run at 30-40% effective utilisation because users over-request compute resources by 2-3x to avoid job failures, wasting significant capacity and money. Existing methods like historical averages, hand-written rules, and LLM agents cannot accurately predict actual resource needs before job submission.

Solution concept

A system that installs on cluster nodes and analyzes source code, submission scripts, and live hardware telemetry to predict actual resource needs (GPU VRAM, memory, CPU, walltime) before job submission, with confidence intervals and failure predictions. The product would also provide live observability dashboards, code-level optimization suggestions, and failure diagnosis with actionable fix recommendations, continuously improving predictions as more workloads run on the cluster.

Who has it

Organizations running large-scale GPU and HPC clusters (national HPC facilities, quant funds, AI labs, manufacturing companies, cloud datacenters) with job schedulers like Kubernetes and SLURM.

How they handle it today

Today, users submit jobs requesting more resources than needed because under-requesting causes job failure and loss of days of work. Cluster operators may use SLURM accounting databases (sacct) for per-user historical averages, hand-written scheduling rules, or frontier LLM coding agents to manage resources, but these methods fail to capture resource needs accurately when workloads change or new job types are submitted.

Why now

Large language models and deep learning have matured to solve this problem more accurately than rule-based systems, and cluster operators face growing pressure to optimize costs as compute infrastructure scales. The financial impact is significant ($8.5M wasted per month on a single national-scale cluster at on-demand rates), making the ROI on solutions clear.