Executive summary
This report presents AI as a layered service system. Small models handle bounded tasks, larger models support demanding work, and specialist capabilities must be coordinated with data, tools and infrastructure. A healthcare analogy makes those roles and their dependencies accessible to investors and practitioners.
The argument is that model scale and accelerator performance remain important, but cannot explain application economics on their own. Data movement, memory availability, routing and orchestration determine whether expensive resources translate into useful work. A strong component can still sit inside an inefficient system.
Key findings
- Different workload classes justify different model and infrastructure tiers.
- MoE specialization creates communication and coordination requirements alongside compute savings.
- Memory and data movement can constrain realized performance even when compute capacity is available.
- Agentic applications make orchestration across models, tools and data a central part of infrastructure efficiency.
The layered-system analogy
Primary-care clinics illustrate the role of smaller models that can handle routine workloads with lower cost and latency. Major hospitals represent the broader capability and higher resource requirements of frontier models. Specialist departments illustrate selective expert activation, while coordination between services helps explain the importance of routing and workflow management.
The analogy also exposes the need for escalation. Efficient systems must decide when a smaller or specialized model is sufficient and when broader capability is required. Using the largest model for every request can be expensive; using a smaller model outside its competence can compromise the application. Matching capability to task is an ongoing systems problem.
Where infrastructure bottlenecks emerge
The report examines blind spots across compute, connectivity, memory and orchestration. Distributed models require synchronization and movement of intermediate data. Inference must store and access weights and context, while agent workflows introduce multiple calls, external dependencies and coordination delays. These demands can limit the value of additional peak compute.
Optimization techniques such as quantization, distillation and speculative decoding alter different parts of this system. Their value depends on workload suitability and integration with the serving architecture. The related research notes explore those techniques in more focused scenarios.
Implications for infrastructure strategy
For practitioners, the report encourages evaluation of complete application paths rather than isolated component specifications. For industry analysis, it draws attention to enabling infrastructure that can be overlooked when discussion centers on model size and GPU shipments. Networking, memory and software coordination influence the ability to monetize deployed capacity.
Assumptions and limitations
The healthcare comparison is an explanatory framework, not a quantitative forecast or evidence that AI systems make decisions like clinical organizations. The report's strategic argument should be distinguished from the specific simulation assumptions and numerical findings in the companion studies.
Source: The AI Frontier, PDF pages 1–19. This summary describes the published report; consult the PDF for figures and references.