Your AI Infrastructure Is a City. Now the Blueprint Takes Shape on Screen.

"AI workload-aware design, validated before deployment, promises to deliver." Illustrated by DALL·E
AI infrastructure design is slow, complex, and unforgiving, where every mistake comes at a cost.
UpScale AI showed how to close that gap with SkyD (Sky Design). This is the capability I think many hyperscaler and neocloud architects have been missing. The next gap to close is the one between the blueprint and the budget.
Every token pays the network tax
Inference has two phases with opposite bottlenecks. Prefill reads the prompt and is compute-bound. Decode writes the answer one token at a time and is memory-bandwidth-bound. The network joins in because a model rarely fits on one GPU. Split it up and the GPUs must trade partial results: per layer for tensor parallelism, usually inside the rack, and across racks for pipeline and expert traffic. Meanwhile the KV cache linking the two phases keeps growing, and moving it adds traffic. Longer context and bigger models mean more of everything.
The concept isn’t the hard part. The scale is: thousands of GPUs, hundreds of switches, thousands of cables and fibres.
No one can manage that manually, yet many still design with rules of thumb. The currency is GPU time; every GPU waiting on the network is wasted spend. Right sizing stops being a skill and becomes a risk, one that decides whether the AI investment pays back.
Think city, not datacenters
- Scale-up is the lifts inside the building - GPUs in a rack, sub-microsecond latency.
- Scale-out is the streets between buildings - Racks linked over Ethernet or InfiniBand.
- Scale-across is the motorway between cities - Sites joined into one AI factory.
If one layer is undersized, the whole city jams.
SkyD plays city planner. You start with the basics: the ports on the compute (such as NVIDIA B300 or GB300 systems) and the network switches. It then applies design rules to size and shape the backend, frontend and management networks. The rules are foundational, such as no oversubscription on the backend while tolerating some on the frontend. Multiple network planes add redundancy and let a two-tier design support far more GPUs.
Seeing the city before it’s built
What won me over is the visualisation.
Designing and sizing an Al cluster is tough, and it has mostly lived in static documents and in people’s heads. SkyD puts it on screen. It turns abstract leaf-spine and superspine topologies into real site maps, with floor layouts, rack elevations and cable routing that shortens fibre runs. A resiliency map shows how a switch, optic or fibre failure would hit bandwidth, so you can plan for faults before they happen.
Add the design rules and the pre-tested, fully documented blueprints, and you have software-aided design that bridges the distance between the first draft and production deployment. It was described as
“Day minus one” tool for architects, which is exactly where mistakes are cheapest to fix.
What you get on both sides of the brain
For the analytical side, you get sizing rules, validated design blueprints and a shorter road from ideation to deployment. For the human side, you get confidence that the design on screen is supported by foundational guardrails, one shared picture for network, infrastructure, facilities and finance teams, and architects spending their time on decisions rather than on checking compatibility and cables.
What was said, what wasn’t, what nobody knows yet
- Said: pre-tested, fully documented architectures that bridge design and deployment.
- Unsaid: the best design on paper still has to be bought, and the one that gets built may be the second or third choice. Cost, lead time and power decide.
- Unknown: is it vendor-neutral, and has its output been validated against a real deployed cluster? How do the rules differ for inference versus model training? How do designs adapt as workloads shift?
The next gap: from blueprint to budget
The logical and physical design output is strong. I’d like equal emphasis on the commercial output: a costed bill of materials, side-by-side options showing cost against performance, and compatibility and lead-time flags, because supply can gate a schedule. Add power and cooling costs over three to five years, phased growth you can fund in stages, and a visible price on each layer of resilience. Close the loop with an export to procurement.
The takeaway
Faster GPUs or extra switches won’t fix AI infrastructure complexity once performance problems appear. You need to understand what data is moving, anticipate how often, “who” and where it is waiting-at scale. AI workload-aware design, validated before deployment, promises to deliver.
Add a commercial view and a design becomes a decision.
- Nicolas Fevrier conducts the whole city like an orchestra: with topology, racks, fibre and failure paths, all playing in time - SkyD by Upscale AI
- Aninda Chatterjee maps out the city’s roads and tiers : a lesson in AI Clusters design - Network Engineer Meet AI Cluster with Upscale AI
- Access all sessions on Tech Field Day AI Infrastructure Field Day 6
May Mun is a guest writer for Mawaːga. You can find her on LinkedIn.


