SCALING INTELLIGENCE: THE GENERATIVE AI HANDBOOK

Research evidence excerpt

SCALING INTELLIGENCE: THE GENERATIVE AI HANDBOOK

being taken, the concern about the economic impact is, as of the time of writing of this Blackbook,

negatively impacting the valuations of per-seat-based enterprise software companies.

While training is compute-bound, inferencing today is bandwidth-bound, which is driving the rise

in new types of inferencing chips and the use of application-specific integrated circuits (ASICs)

at many of the hyperscalers. For more information, see the chapter titled “Generative AI 201:

Understanding the Cost of Inferencing” of this Blackbook.

REASONING MODELS ARE A After around two years of intense scaling of the foundational model, reasoning models came into

STEP ON THE JOURNEY TO the spotlight with the release of OpenAI’s (private) o1 and DeepSeek R1 in late 2024 and early

BETTER, MORE CONSISTENT 2025, respectively, and we deep-dive into reasoning in the chapter titled “Generative AI 301:

RESULTS

Reasoning” of this Blackbook. In contrast to the foundational model, a reasoning model can “think

out loud” in a step-by-step manner to solve a problem. This is helpful in complex scenarios that

require multi-step reasoning and execution (e.g., agents), where the earlier generation models

tend to “jump to a conclusion” with wrong guesses.

Technically, reasoning models are built on top of and will not replace foundational models, but

where the compute is spent shifts. To build a reasoning LLM, while there is no consensus, the

prevailing methods are to either feed more reasoning data during the post-training stage, or

ask the model to “reason longer” during inference. These are highly curated reasoning data sets

that lay out human thought processes in solving particular problem sets, and often need to be